We explore the effects of hierarchical structures on organizational learning by critically examining the traditional belief that flat hierarchies are superior for learning and innovation. We challenge this notion by introducing varying hierarchical influences across hierarchical layers. By employing computational modeling, we demonstrate that the impact of hierarchy on organizational learning is more complex than previously understood. Our findings reveal that without taking into account hierarchical influences, contrary to conventional wisdom, tall structures consistently outperform flat structures. Flat organizations outperform tall ones when a significant hierarchical influence is present. We also show that lateral and random ties among individuals can significantly enhance learning performances of flat organizations. Our study establishes boundary conditions for the effectiveness of flat versus tall organizations, providing new insights into the interplay of hierarchy, organizational structure, and learning dynamics.