Abstract: Presently, digital data gets exponentially raised owing to an increase in number of data channels which generate and distribute data, load balancing techniques are developed for handling big data in real time. Though the cloud environment offers effectual services, it faces some serious issues of load balancing where the improper distribution of load results in degraded overall processing performance. This paper presents a novel Cross Entropy with Glowworm Swarm Optimization Algorithm based Load Balancing (CEGSO-LB) Technique for Distributed Big Data Systems. The aim of the CEGSO-LB model is to reduce the overall processing cost and schedule the load on the VMs proficiently. The presented CEGSO algorithm incorporates the basic concepts of CE method and GSO algorithm. The CE concept is integrated into the GSO algorithm to improve the efficiency in attaining global solutions and eliminating local optima problem. The presented model is implemented to examine the results under varying sizes of synthetic datasets and varying number of Virtual Machines (VMs). The experimental results guaranteed the betterment of the CEGSO-LB technique interms of distinct aspects namely Average Load, Average turnaround time, Average response time, CPU utilization, memory utilization, reliability, average execution time, makespan, and average throughput.