The nasty details of the custody battle between Halle Berry and Gabriel Aubry went public this week, but sources tell PEOPLE things really started going downhill around the holidays.

Berry, 44, reportedly became alarmed when the couple's 2-year-old daughter Nahla became hysterical when Aubrey, 34, came to pick her up for a visit a few days before Christmas. A source said Nahla "was crying and kicking and screaming. She was hanging onto her mother's legs so she didn't have to go."

The source continued to tell PEOPLE that when the 'Monsters Ball' actress tried to calm down her daughter, Aubry "got Nahla and forced her into the car." Berry was extremely upset by the incident, and sources close to the Academy Award winner claim similar incidents have occurred since then, including one occasion where Nahla seemed "traumatized" after spending time with her father.

Berry has spoken out about having "serious concerns for her daughter's well-being while in the care of her father for any extended period of time and is prepared to take all necessary steps to protect her." Contrarily, Aubry has claimed that Berry is the one with the temper, and supposedly has a handful of voice mails, e-mails and text messages displaying fits of rage and name calling by the actress.