Position rack in center of oven and preheat to 350ºF. Butter a loaf pan and line on all sides with parchment paper.
In a large bowl, whisk together oil, brown sugar, eggs, buttermilk, and vanilla. In a medium bowl, sift together cocoa, sweet rice flour, almond flour, oat flour, tapioca flour, baking powder, baking soda, and sea salt, adding back anything that gets caught in sifter. Stir flour mixture into egg mixture until combined. Stir in chocolate and figs.
Scrape batter into pan and bake until a toothpick inserted near center comes out with moist crumbs, 55-65 minutes.
Let cake cool 15 minutes, then remove to a cooling rack to cool completely.
To make ganache, heat cream in small saucepan to a simmer. Place chocolate in small heatproof bowl and pour hot cream over. Let sit 1 minute, then add salt and vanilla and whisk smooth.
Let ganache sit until thickened slightly, 20-30 minutes, then pour over cake. Let sit until ganache is mostly set, 10-20 minutes, then decorate top with sliced figs, almonds, and coarse salt. Cut cake into slices using a large, sharp chef’s knife dipped in hot water and wiped clean between each cut.
Store cake at room temperature for up to 1 day or refrigerate for up to 4 days.