Brad Pitt 'Wanted to Quit' Legends of the Fall, 'Wasn't Pleased' with the Final Film, Says Director

Ed Zwick recalled his experience directing the film in his upcoming memoir 'Hits, Flops, and Other Illusions: My Fortysomething Years in Hollywood'

Ed Zwick is ready to pull back the curtain on his work with Brad Pitt.

In an excerpt from his upcoming memoir, Hits, Flops, and Other Illusions: My Fortysomething Years in Hollywood, published by Vanity Fair on Tuesday, the Blood Diamond director, 71, reflected on directing Pitt, 60, in Legends of the Fall.

Zwick wrote that the 1994 film was "in limbo" after Tom Cruise, who was in talks to play Tristan Ludlow, dropped out due to issues with the character's "ethics."

"I never gave up hope on getting Legends made, however, and was always looking for the right actor to play Tristan," wrote Zwick. After meeting with Pitt, Zwick observed that he "had a genuine passion for the script and a strong attraction to the character."

"Growing up in rural Missouri, he had known men like Tristan, he said. When he left the meeting, I felt I had found the right actor. I was more determined than ever to push it over the line," Zwick wrote.

According to Zwick, Pitt's feelings about the film changed after a table read. He recalled Pitt's agent calling the studio saying he wanted to quit and producer Marshall Herskovitz "talking him off the ledge."

Zwick wrote that "it was the first augury of the deeper springs of emotion roiling inside Brad," expressing that he could be "volatile when riled."

Following the start of -

Toni, Tamar and the Family Return to TV ,...

filming, production issues that included spending an "overage of a million dollars in costumes" occurred, and Zwick said, "Brad's anxiety about the movie never quite went away."

"Some actors have problems with authority, but just as many directors are threatened when intelligent actors ask challenging questions that reveal their lack of preparation. Both are right, and both are wrong," wrote Zwick.

He described incidents in which the two would disagree about the character, including one in which he "started giving Pitt direction out loud in front of the crew," causing Pitt also to be "loud" and tell him to "back off."

"In his defense, I was pushing him to do something he felt was either wrong for the character, or more 'emo' than he wanted to appear on-screen. I don't know who yelled first, who swore, or who threw the first chair. Me, maybe? But when we looked up, the crew had disappeared," wrote Zwick.

Zwick wrote that the crew "grew accustomed to our dustups" but that he and Pitt would "make up" after each "blowup."

Never miss a story — sign up forPEOPLE's free daily newsletterto stay up-to-date on the best of what PEOPLE has to offer, from celebrity news to compelling human interest stories.

"It was never personal. Brad is a forthright, straightforward person, fun to be with and capable of great joy. He was never anything less than fully committed to doing his best," wrote Zwick.

PEOPLE has reached out to Pitt's representatives for comment.

Michael C. Hall to Return for ‘Res,...