Why Passive Investors Should Become Stock Pickers

When I started investing 20 years ago, I was terrified of the idea of investing in individual stocks. My parents never taught me about the stock market, and all that I knew was what I read in books. So when I started investing, I invested in stock and bond mutual funds.

I did so poorly at first. An “advisor” at a local bank with a brokerage arm steered me into a few expensive mutual funds with front loads (fees you pay just for the privilege of giving a mutual fund your money). The funds were actively managed, not index funds. That meant that on top of the front load, I was also paying a lot in fees each year so that a manager could pick stocks the fund would buy. The fund never beat the S&P 500. So basically, I paid a mutual fund a ton of money to underperform the market.

It didn’t take long for me to conclude something was very wrong with this picture. With the help of some great books, like Rick Ferri’s All About Asset Allocation, I concluded that passive investing was the best way for me to build wealth in the stock market. This lead me to a fanatical attack on fees and managed funds. I moved most of my money to low cost index funds. You can read about my views on mutual fund fees, the hidden costs of mutual funds, and how costs can eat away at your retirement.

As I learned more about the market, I eventually concluded that picking individual stocks was like chasing fool’s gold. You may be able to beat the market for a time, but like the great Bill Miller (who’s Legg Mason fund beat the S&P 500 for about 10 years and then cratered), eventually your luck will run out. So I stuck with mutual funds, most of them index funds.

Today, my perspective has changed significantly. While most of my investments are still in low cost mutual funds, over the past two years I’ve started investing a portion of our money in either individual stocks or industry-specific ETFs. There are two big reasons why I’ve taken this approach, which I’ll discuss in a moment. But first, let’s talk about passive investing.

What is passive investing

Passive investing is about a lot more than index funds. In fact, I’d argue that passive investing is less about what you invest in than how you invest.

For example, I know plenty of people who invest all of their portfolio in low cost index funds. So far, so good. But when a bear market comes along, they sell their index funds and stash their money in bond funds because it makes them feel safer. When the market is going up, they do the opposite. This is not passive investing.

On the contrary, I’d argue that you can be a passive investor by picking a single stock and sticking with it through good times and bad. You’d have to pick the right stock in a company that is very diversified, like Berkshire Hathaway. But to me, such an investor would be far more passive than one who jumps in and out of index funds as the market rises and falls.

Why individual stocks have a place in a passive investor’s portfolio

As I’ve had more experience investing and now manage a much larger portfolio than I did 20 years ago, my views on investing have changed. Today I own stock in several companies (Apple, Cisco, Citi, Verizon, Pepsi, Berkshire) and an industry-specific ETF (ITB-Housing). I’ve made this move for two reasons.

First, in taxable accounts, you have far more control over capital gains with individual stocks than you do mutual funds. Unless I sell some of my investments, I pay no capital gains tax. While you can invest in tax efficient mutual funds, you still don’t get the same level of control. The same is true for dividends.

If you have $10,000 to invest, the tax issue may not matter to you right now. But as your portfolio grows to 6 and even 7 figures, taxes will become huge. And that’s particularly true if you are in one of the top tax brackets.

Second, it’s cheaper to invest in individual stocks than it is to invest in mutual funds or ETFs. With ETFs and index funds, you can find some really inexpensive ways to invest. Vanguard’s S&P 500 fund, for example, costs just 5 basis points a year (just $5 for every $10,000 invested). But every dollar counts, and with individual stocks there are no mutual fund management costs. You do have to pay a brokerage fee when you buy the shares, but the best online discount brokers cost just a few dollars per trade.

In my case, I’ve been able to lower my overall expense ratio from 48 basis points to just 28. Twenty basis points (.20%) may not seem like a lot, but over a lifetime of investing those costs add up to a significant amount.

But what about diversification?

I was nervous when I first starting investing in individual stocks. But over time I’ve come to really enjoy it. I still have the vast majority of our money in low cost mutual funds and ETFs. But the portion I have in individual stocks has become an important part of our investment plan. Our costs are down, and our taxes are down.

All of that being said, investing in individual stocks is not for everybody. For beginners, I’d stick with low cost mutual funds, ETFs, or something like Betterment (an excellent option that has really come down in price).

If you’re a passive investor who still buys individual stocks, leave a comment to let us know why you’ve taken that approach and how you’re doing. If you think buying individual stocks is a big mistake, we’d like to hear from you, too!

Topics: Investing

17 Responses to “Why Passive Investors Should Become Stock Pickers”

  1. I had the same experience with buying a front load fund. It was an expensive learning process and it turned me on to Vanguard and mutual funds with low expense ratios.

    I don’t have many individual stocks, but I am investigating purchasing more in the long run – for similar reasons as you. Right now I hold GE and Berkshire but I’m looking at adding some dividend stocks so I can build more long term revenue streams (and enjoy the low tax rates for dividends). I think dividend investing, when done well, can be a huge asset in your portfolio.

  2. Larry

    What percentage of your assets do you keep in individual stocks?

    And can’t you get a similar tax benefit by contributing the max to a Roth IRA each year?

    • Rob Berger

      Larry, right ow individual stocks make up a pretty small portion of our investments, but they are growing. As for a Roth IRA, our income disqualifies us.

  3. gordon

    Rob – good article but you messed up your costs by a factor of 10.

    The Vanguard S&P 500 isn’t $5/$1000 invested. It is fifty cents. It isn’t 0.5% of the invested assets, but 0.05%. Big difference.

  4. TempleRheum

    “Vanguard’s S&P 500 fund, for example, costs just 5 basis points a year (just $5 for every $1,000 invested).”

    Did you mean to say $5 for every $10,000 invested?

  5. Steve

    There is no way you could invest in all of the individual stocks in the S&P 500 cheaper than Vanguard is doing it. That’s like trying to out-Walmart Walmart or trying to out-hamburger McDonalds. They simply operate at economies of scale orders of magnitude higher than any individual investor (seven figure portfolio or otherwise).

    • Rob Berger

      Steve, agree completely that you couldn’t invest in all of the S&P 500 companies for less than a Vanguard fund. But you can invest in a select number of companies and lower your overall investing costs.

  6. Looks like this guy doesn’t respond to comments, but I’m curious as to what percentage of your portfolio individual stocks make up? I bet it’s no more than 5-10%.

    I did some research and it seems like what you are doing is a little redundant. From the list you gave, all 6 of the companies you invest in are in the top 37 holdings of Vanguard Total Market. Vanguard Total Market holds the following percentages:

    Apple – 3.63%
    Cisco – .74%
    Verizon – .70%
    Citi – .70%
    Pepsi – .68%
    Berkshire – .43%

    Add these all up and you get 6.87% Let’s say you’re holding 70/30 stocks/bonds. If 100% of your stock investments are in Vanguard Total Market, then 4.8% of your entire portfolio is already invested in these funds that you have ‘picked.’ So you’re already invested your portfolio in 5% of these companies now you add whatever your individual stock portion is. Make sense?

    As for passive investing, the key to passive investing is staying the course. If your desired AA is 80/20 stocks and bonds, when a bear market hits you should re-balance accordingly and buy more stocks. I think most passive investors know this. I don’t agree with you that most passive investors do the opposite, I think that’s what active investors do..

    • Rob Berger

      Thanks for the input. I don’t own the Vanguard Total Market, but there’s no doubt that there is overlap between my individual holdings and mutual funds. But that doesn’t concern me. I like the dividend exposure and lower costs of individual stocks. Also, I didn’t say that most passive investors sell in a down market and buy in a up market. But study after study tells us that many do.

  7. Andrew

    I started investing about three years ago, at the age of 20, and skipped experiences like the expensive lessons Rob and some of the other people commenting have thanks to my grandfather who introduced me to vanguard. This is now my third year maxing out my roth ira in mostly a tr fund but my individual stock holdings in my brokerage account have outperformed my IRA with such significance, even with a big roth head start as I entered during a huge correction, that I’m considering making individual stocks of my choosing a considerable portion of my Roth. Maybe 30% while I can still carry the risk involved being that I have a lot of years to weather a storm before going to maybe 5% individual holdings in my latter years – depending on their growth. Has anyone here personally, or know someone they can speak of, had long term success with a high initial allocation towards individual holdings? Without diving into what makes a good buy and hold forever stock. By the way, a 401k is not an option for me now but it will be in the future; hopefully with a self directed brokerage account option!

  8. Jason Brand

    I have stocks in a few very large companies whose dividends are well above what I can get in short-term fixed income vehicles. The companies are in energy, consumer staples, and health care, and their dividends have grown steadily for many years. I don’t intend to sell any of them. I also have an industry ETF and am contemplating a second.

  9. I have owned both an all-stock portfolio, and at other times, an all index-fund portfolio.

    However, it sounds like your portfolio is a mix of funds and stocks. In that case, you really just have a fund portfolio that is slightly skewed from the indexes you are tracking. Other than the “fun” factor, I see no value in your approach…..over time there is likely to be little difference….you still effectively just own a fund.

    My own experience is that a low-priced fund portfolio is better than an all-stock portfolio. It is less hassle, has less volatility and risk, and lowers your chances of selling and buying all the time, and thus, wrecking your returns.

  10. Good blog. I also invest in mutual funds and individual stocks. 79% funds but moving to 60%. The advantage of individual stocks is uiu can buy them at the right time and sell if or when you want. I drink a beverage from fizz ( stick symbol) and it went from 17.00 to 105 in a bit more than a year. Also bought nintendo before switch cane out and now up 50%. All sticks won’t be winners but you only need one or two big wins to win big and best mutual funds . And it’s fun for me. I read books and do done research but only about two hours worth per stock I buy. So yes, stocks are good to buy individually and in a sense, you have your own Personal mutual fund . and so far for me, better returns by far than sp500. so I say do both if you have the desire , if not, buy low cost mutual funds only.

Leave a Reply