Cloud Computing is a robust model that allows users and organizations to purchase required services per their needs. The model offers many services such as storage, platforms for deployment, convenient access to web services, and so on. Load Balancing is a common issue in the cloud that makes it hard to maintain the performance of the applications adjacent to the Quality of Service (QoS) measurement and following the Service Level Agreement (SLA) document as required from the cloud providers to enterprises. Cloud providers struggle to distribute equal workload among the servers. An efficient LB technique should optimize and ensure high user satisfaction by utilizing the resources of VMs efficiently. This paper presents a comprehensive review of various Load Balancing techniques in a static, dynamic, and nature-inspired cloud environment to address the Data Center Response Time and overall performance. An analytical review of the algorithms is provided, and a research gap is concluded for the future research perspective in this domain. This research also provides a graphical representation of reviewed algorithms to highlight the operational flow. Additionally, this review presents a fault-tolerant framework and explores the other existing frameworks in the recent literature.