A recent U.S. Travel Association survey found less than half of U.S. workers are using their vacation days — fewer than ever before. Millennials, now the largest generation in the workforce, are the least likely to take days off.

About 70 percent of millennials surveyed this year by Alamo Rent-A-Car said they were subject to so-called “vacation shaming” by colleagues.

“You know, you’re getting ready to go out on your big trip, and they make the comment, “Oh, that must be nice. I’ll be here. I have too much on my list,” said Cait DeBaun with Time Off, a campaign by the U.S. Travel Association. She said the problem of unused vacation days could get even worse, because millennials are increasingly managers.

“And so we’re setting the tone for other colleagues and employees,” Debaun said. “And when we don’t take our time off, it’s going to continue this trend in a downward way.”    

Former U.S. Labor Secretary Seth Harris said millennials are driving the trend now because they struggled to find jobs during the recession and they’re anxious to hold onto them.

“They’re scared and they’re worried about their futures,” Harris said. “There’s a lot of evidence that that negative impact that hit them early in their careers could last throughout the rest of their work lives.”

Even if you do take a vacation, it can be hard to fully unplug from work, especially for tech-obsessed millennials.