Combining photovoltaic arrays with batteries, heat pumps and thermal storage further decarbonizes the heating sector. When evaluating the performance of such systems, the parameters are either fixed, or based on generic ranges, or characterized by precise distributions inferred from limited information. These assumptions result in suboptimal designs, for which the actual performance differs drastically from simulations. To address these limitations, we consider the effects of limited information (epistemic uncertainty) on the natural variability (aleatory uncertainty) through probability-boxes. First, we performed a robust design optimization on the natural variability of the levelized cost of exergy, followed by a sensitivity analysis on the effects of limited information on the optimized designs. This paper provides the least-sensitive designs to natural variability and effective actions to reduce the effects of limited information. The results indicate that a photovoltaic-battery-heat pump configuration achieves higher robustness towards aleatory uncertainty than a photovoltaic-battery-gas boiler configuration. To determine the true-but-unknown performance and robustness of the optimized designs, clarifying the grid electricity contract and adopting specific energy demand profiles are the main actions, while considering generic technology models contributes little to the epistemic uncertainty.