In a medium saucepan over medium heat, bring the almond milk to a simmer.
In a medium bowl, whisk the egg yolks with ½ cup of the powdered sweetener and the salt until pale yellow.
Slowly whisk about half of the hot milk into the yolks to temper them, then return the yolk/cream mixture back to the saucepan, whisking continously. Cook until thickened, whisking continuously.
Remove from heat and whisk in the butter and vanilla extract. Let cool to room temperature.
In a large bowl, beat the heavy whipping cream with the remaining ¼ cup powdered sweetener until it holds stiff peaks. Fold the pastry cream into the whipped cream until no streaks remain.
In a small saucepan over medium heat, bring the cream to a simmer. Remove from heat and add the chopped chocolate. Let sit 5 minutes to melt.
Add the sweetener and vanilla and whisk until smooth.
Line the bottom of an 8x8 inch pan with graham crackers, cutting to fit as needed. Spread half the filling over the graham crackers. Repeat with another layer of graham crackers and the remaining filling.
Top with one more layer of graham crackers and pour the chocolate ganache over top, spreading to the edges. Refrigerate at least 4 hours to firm up before serving.