Purpose The Monte Carlo (MC) algorithm has been widely accepted as the most accurate algorithm for dosimetric calculations under various conditions in radiotherapy. However, the calculation time remains an important obstacle hindering the routine use of MC in clinical settings. In this study, full MC three-dimensional dose distributions were obtained with the inputs of the total energy release per unit mass (TERMA) distributions and the electron density (ED) distributions using a convolutional neural network (CNN). A new Dose-mixup data augmentation routine and training strategy are proposed and applied in the training process. Attempts were made to reduce the calculation time while ensuring that the calculation accuracy is comparable to that of the MC. Methods Datasets were generated via the MC with random rectangular field sizes, random iso-centers, and random gantry angles for head and neck computed tomography (CT) images with Mohan 6-MV spectrum photon beams. 1500 samples were generated for the training set, and 150 samples were generated for the validation set. The T-MC Net model was obtained with the Dose-mixup data augmentation routine. The new CTs were used to test the performance of the model in the rectangular fields and the intensity-modulated radiation therapy (IMRT) fields. The mean +/- 95% confidence interval of gamma pass rates were calculated. Results For 150 rectangular field test samples, the 1%/2 mm, 2%/2 mm, and 3%/2 mm criteria gamma pass rates were 90.11% +/- 0.65%, 97.65% +/- 0.31%, and 99.16% +/- 0.19%, respectively. For the 100 IMRT field test samples, the 1%/2 mm, 2%/2 mm, and 3%/2 mm criteria gamma pass rates were 96.48% +/- 0.28%, 99.14% +/- 0.10%, and 99.63% +/- 0.06%, respectively. For the 7-fields IMRT plan, the 1%/2 mm, 2%/2 mm, and 3%/2 mm criteria gamma pass rates were 97.06%, 99.10%, and 99.52%, respectively. For the 9-fields IMRT plan, the 1%/2 mm, 2%/2 mm, and 3%/2 mm criteria gamma pass rates were 98.16%, 99.61%, and 99.89%, respectively. Conclusions The feasibility of calculating dose distribution using a CNN with the TERMA three-dimensional distribution and ED distribution was established. The dosimetric results were comparable to those of the full MC. The accuracy and speed of the proposed approach make it a potential solution for full MC in radiotherapy. This method may be used as an acceleration engine for the dose algorithm and shows great potential for cases where fast dose calculations are needed.