Elder broke through Augusta’s race barrier at the 1975 edition of the famous competition.
He was honored at the 2021 Masters, joining Jack Nicklaus and Gary Player as an honorary starter in the ceremonial first tee shot to begin the tournament.
Elder told CNN in 2015 that making his Masters debut in Augusta, Georgia, was a “very nerve-racking” experience.
“I was shaking so badly, I did not know if I was even going to be able to tee up the ball,” he said.
“How I got through it I do not know, just with the help of the Almighty I got there and was able to put my ball on the tee.”
Elder broke into the game in the US during the 1960s and 70s, a notoriously volatile period for race relations.
In doing so, he became one of golf’s most recognizable faces, but that came with its problems.
At one tournament, he had been forced to change in the parking lot after being refused entry to the clubhouse; during another, his ball had been hurled into a hedge by a spectator.
Even in the year leading up to his breakthrough Masters appearance, Elder received intimidation and threats, much of which warned him not to travel to Georgia — some of it made plain to him what would happen if he did.
“It was frightening. You try to eliminate the possibility of anything happening,” he said.
“That was part of the reason for renting two houses during the Masters week. The logic behind that was the fact we did not want the people to know where I was staying.”
Elder had qualified for the Masters courtesy of a tournament victory in Pensacola, Florida.
But such was the negative reaction to the first of his four PGA Tour titles, he considered whether accepting the invitation to the Masters was the right thing to do.
“I did consider not going,” Elder says. “It was on my mind and I think the reason why I had thought about it was because it had been so difficult qualifying for the Masters.
He said it took him about a week or so to make up his mind. “I knew it was something I had wanted ever since I came on to the Tour.
“I think the reason why was there had been so much talk about no Black man playing at Augusta, and after all that, I wasn’t going to qualify and not go.”
Though he went on to miss the cut, he returned a further five times, tying for 19th place in 1977 and 17th in 1979.
Elder would go on to become the first Black man to represent the United States Ryder Cup team when he played in the 17-11 win over Europe in 1979.
Renee Powell, the second Black woman to compete on the Ladies Professional Golf Association (LPGA) Tour and Elder’s friend, posted on Facebook that she was “very saddened” by the news.
“Lee had called me last week about doing a project together this summer,” Powell said.