Why Wealthy People May Want to Take Social Security at 62

I’m always surprised how many high-net-worth individuals and families overlook one of the most important parts of retirement planning: the enormous amount of taxes that they may have to pay on their Social Security benefits.

It’s understandable. They aren’t depending on Social Security as the foundation of their retirement like most Americans do. For many retirees, boosting that monthly check can make or break their income plan. But those who have saved $1 million, or multimillions, in their portfolios have opportunities to do more with their money than most people would imagine. These opportunities include reducing income taxes, reducing taxes on Social Security benefits, lowering Medicare premiums, deducting investment fees and more.

The question you should be asking yourself is this: Am I inadvertently on a course to allow decades of hard work and savings to be eaten away in taxes?

Subscribe to Kiplinger’s Personal Finance

Be a smarter, better informed investor.

Save up to 74%

Sign up for Kiplinger’s Free E-Newsletters

Profit and prosper with the best of expert advice on investing, taxes, retirement, personal finance and more - straight to your e-mail.

Profit and prosper with the best of expert advice - straight to your e-mail.

Sign up

My firm reviews hundreds of portfolios each year. It’s frustrating to watch people who saved diligently all their lives find themselves at retirement in a position that I call the “government plan.” How and where they’ve saved money through the years now dictates how much money they’ll pay in income taxes — possibly unnecessarily. They’re in this position because they didn’t think about how and when they would pull money out of these accounts in retirement.

Social Security Rules of Thumb Aren’t Perfect

These days, too many people in the news and the financial advice world stress taking Social Security at full retirement age to get their full benefit amount, and the fact that by waiting even longer to file, the benefit can grow 8% a year from full retirement age to age 70. Based on my Social Security experience, this one-size-fits-all advice may not be appropriate — especially for the wealthy.

The problem we see is that everybody wants to boost their benefits on the front end. But if there is no plan to boost those dollars on the back end, by keeping more of your Social Security dollars for yourself instead of giving them to Uncle Sam, then what’s the point? For many, by the time they realize they’re going to be giving what could be 20% to 30% of their Social Security away in the form of taxes, it’s too late.

One Wealthy Couple’s Revealing Story

Indeed, while conventional wisdom says that, if possible, you should hold off and file for bigger Social Security benefits at age 70 — something high earners in many cases can manage — there are a growing number of couples who would be better off filing at age 62 and using that income to preserve and build their nest egg. Here’s a recent example to illustrate the concept. We counseled a family where the husband was retiring at age 62. With the paycheck no longer coming in, they were both about to find themselves in the lowest tax bracket they had been in since their first jobs: the 10% bracket.

The issue that immediately came to light is they, like many Americans, will look to pull from their IRA and 401(k) in retirement. These are generally the most significant accounts in terms of amounts saved through the working years. If this family didn’t turn on Social Security at age 62, they would need to pull heavily from pretax retirement accounts. Based on the monthly distribution rate needed to maintain their budget, those dollars — taxed at current income tax rates — would immediately put them into a higher tax bracket (potentially the 22% bracket under 2018 tax rates).

If they were to go ahead and take their Social Security payments at age 62, the monthly distribution amounts needed from their retirement savings accounts would be substantially smaller. As we’ll demonstrate, this couple’s story shows how heavily draining one’s retirement accounts early in retirement can result in lost opportunities for compounded growth of assets over a 20- to 30-year retirement.

If they were to take their Social Security at age 62 — while in a 10% tax bracket from age 62 to 70 — the amount of tax they would pay on those Social Security benefits would be minimal, possibly even zero. There are cases where our families who have done a great job of saving across many accounts with true tax diversification in their portfolio will pay no tax on their Social Security benefits for a good majority of their retirement (based on 2018 Social Security Base Amount limits, which have been in effect since 1983).

How Taking Social Security Early Can Pay Off

If this family were to delay Social Security payments from age 62 to age 66, it’s estimated they would be deferring nearly $146,000 in Social Security payments over that period. They would be forced to pull money from their retirement accounts to live on, paying a projected $51,372 in federal taxes from age 62 to 66.

Instead, if they elected to take their Social Security at age 62 at a 10% bracket, the tax savings on that $146,000 would be substantial. By drawing early and minimizing taxes on retirement and brokerage funds, the family would owe $9,768 in taxes during the same period. That’s a savings of $41,604 in taxes at the beginning of their retirement career.

To take it a step further, when we reviewed what their retirement account balances would be when comparing the same withdrawal rate, using the same rate of return on their assets (a portfolio weighted average return of 4.47%). The estimated numbers* spoke for themselves.

• $122,000 more at age 65: If they were to take Social Security at age 62 versus age 66, their combined account balances at age 65 are estimated at $1.37 million. If they were to defer Social Security until 66, their combined account balances at age 65 are estimated at $1.248 million. That’s a $122,000 difference: $122,000 more in their bank account at age 65 if they started taking Social Security earlier.
• $144,000 more at age 75: We reviewed further, to age 75. By taking Social Security at age 62, their overall account balance would have totaled an estimated $1.53 million at 75. If they waited to take benefits until 66, their account balance at 75 came in at $1.386 million. That’s $144,000 more in their bank account by taking Social Security earlier.
• $100,000 more at age 85: Now, looking to age 85: By taking Social Security at the early age of 62, their account balance at age 85 would total $1.39 million, versus $1.29 million were they to take Social Security later, at age 66. That’s a $100,000 difference.
• $44,000 more at age 95: Finally, we looked at a 30-year horizon to age 95. If they were to take their Social Security at age 62, their balance would total about $1.14 million at age 95, whereas if they were to take Social Security at age 66, the balance would total $1.096 million at age 95. That's a difference of $44,000, still in the positive.

If you’re like me, I want $100,000+ more in my retirement accounts in my early 60s through my mid-70s, when I'm still active and able to travel and enjoy the fruits of my labor. Ultimately, claiming at age 62 based on their current expenses potentially nets them $44,000 more in late life.

The Bottom Line: Be Thorough

As you can see, Social Security is a complicated topic. Be informed. Talk to a financial adviser — a retirement specialist — about all the rules and strategies that apply to claiming your benefits. You also may wish to include a tax professional and/or an estate attorney in the conversation. Just don’t ignore this important retirement issue. Decisions about when to file and how to integrate Social Security into every aspect of your financial plan shouldn’t be treated as trivial or an afterthought.

You’ve worked for years paying into the system — isn’t it time these dollars start working for you?

* These figures take into account federal taxes using the standard deduction, taxes on Social Security, RMDs, COLAs and average inflation.