(CNN) — After four decades, William “Buddy” Carter — one of the longest serving White House butlers — is set to retire at the end of the week, three people familiar with the matter tell CNN.

Carter’s career began 47 years ago at Blair House, the presidential guest house across the street from the White House, where he served as part of the staff for 15 years before heading to the White House Executive Residence as one of just six butlers. Carter would remain a treasured member of the residence staff, assisting presidents and first ladies and their families from Ronald Reagan’s administration until today.

“For 47 years, across 10 administrations, Buddy Carter was integral to the lives of US Presidents, their families, and their home, the White House,” President Joe Biden and first lady Jill Biden said in a statement.

“We are enormously grateful to Buddy for his kindness, his service, and the times that we shared together — he performed a demanding job with excellence and joy for nearly half a century, always ready to lend a hand. Though it is difficult to imagine life in the White House without Buddy, we wish him all the best on a well-earned retirement, and many happy years ahead with Carol and their beloved children and grandchildren.”

Carter’s decision to retire, says one of the people familiar, was a personal one and prompted by a desire to spend more time with his family. A small, private party with staff will be held Thursday at the White House and the Bidens will hold a small, private lunch for him and his family as part of his departure, a White House official told CNN.

“The history that Buddy has witnessed through the years is astonishing,” said one of the people, who worked with Carter for several years at the White House. “His eye for knowing how the White House works is unparalleled — no one knows it better than Buddy.”

That person said Buddy’s help with events such as State Dinners was incalculable, as he knew which parts of the different rooms had the best sight lines, or where a floral arrangement might work to provide cover for the entry or exit for a dignitary or celebrity.

Carter, like many loyal White House staff in the residence, is beloved by former first families. He was one of two White House residence staff members who Melania Trump took with her on her government jet to the funeral in Texas of former first lady Barbara Bush in 2018. The other was retired head Maître D’ George Hainey. Carter and Bush were close, and the Bush family made sure to invite him to the solemn occasion.

Carter was also a wedding guest when both Chelsea Clinton and Jenna Bush got married.

When Donald Trump was president, Trump once held up the start of the annual Congressional Ball at the White House, just so he could be present to sing “Happy Birthday” to Carter as a surprise cake was brought out on the State Floor, a person who attended the event told CNN.

During Bill Clinton’s tenure, Carter was often the first staff member the former President saw each day, as Carter would bring Clinton his morning coffee and newspaper.

“Barack and I are so grateful to have had his warmth, charm, and dignity by our side for eight of those years,” former first lady Michelle Obama tweeted Friday, along with a photo of her and former President Barack Obama with Carter. “Congratulations, Buddy!”

A first family’s unique living situation in the White House breeds a special relationship with the 90 or so members of the residence staff — particularly the butlers, who assist them with preparing and serving meals, tidying up, keeping them on task and on time, and fulfilling requests at all times, day and night — and above all else, doing so with discretion.

“Buddy has always been caring and loyal,” one of the people who worked with him said. “And that’s what makes him so special. He’ll be missed.”

This story has been updated Friday with reaction from former first lady Michelle Obama.

CORRECTION: An earlier version of this story incorrectly attributed a statement from the White House on Carter’s retirement.

™ & © 2021 Cable News Network, Inc., a WarnerMedia Company. All rights reserved.

Join our Newsletter for the latest news right to your inbox