In the current study, the influences of adding graphene nanosheets (GNSs) on the elastic characteristics of concrete are examined using the Mori-Tanaka micromechanical model. By the use of the base concrete hydration equation, the role of hydration time in the Young’s modulus and Poisson ratio of GNS-filled concrete is analyzed. Also, the formation of GNS agglomeration as one of the most major microstructural features of composite materials containing nano-sized particles is considered in the micromechanical simulation. Significant contribution of the hydration time to the elastic properties of GNS-reinforced concrete is confirmed. Generally, the addition of GNSs can help to enhance the concrete elastic modulus and decrease the Poisson’s ratio. The results show that well-dispersed GNSs lead to the enhancement of mechanical performance of the concrete, whereas GNS agglomeration causes significant reduction of elastic modulus.