In a medium-sized bowl, combine the white whole wheat flour, unbleached all-purpose flour, baking powder, baking soda, cardamom, allspice, and nutmeg. Add the flour mixture to the creamed butter mixture, alternating with the sweet potatoes. Beat until just combined, then add in the 2 tablespoons of milk and stir on low. Allow the mixture time to hydrate so let it rest for 10-15 minutes. In the meantime fix the streusel topping.