Many companies are becoming more flexible with when and where their employees can work, but it’s the direct managers and supervisors who really set the tone on remote work for their teams – and not everyone is crazy about it.

While some managers are fine with their direct reports working remotely, others would prefer more on site “face time” with their workers.

They already have been put through their paces after pivoting overnight in 2020 from managing an in-person team to an all-remote one. In a recent client survey by workplace consulting firm Gartner, more than 40% of managers said they’d been feeling higher levels of stress and were logging more hours than before the pandemic.

So how are they really feeling about having to pivot yet again and manage a team that will only be in the office some of the time?

Here’s what several told CNN Business.

Bring it on

Robert Allenby is the managing partner of a midsize law firm in San Diego. And his firm plans to let most staff – not just the lawyers – work from home a couple of days a week, or more if they don’t have to be in the office for trial preparation or other all-hands-on-deck efforts.

“We’ve gone through the looking glass. So many of the fears people had [about letting employees work from home] have proven not to be worthwhile,” Allenby said. “We have not lost productivity or profitability. More importantly, we have gained an improved morale.”

He’s 55, and says he is surprised that sometimes younger professionals are less amenable to remote work than their age might suggest. “There are younger lawyers who even to this day are butts-in-the-seats kinda guys. And you’re like, ‘Dude, why? Who cares?‘“That said, Allenby notes, it is important to be proactive about integrating and mentoring new hires in person. “It’s our job to make them comfortable.”

Robert Allenby in his home office with his dog Anna in San Diego, California.

It’s aggravating having to chase people down

Lori Smith supports greater flexibility for employees, but said one downside of remote work was having to always chase select team members down.

Smith, who was managing 40 assistants to financial advisers at a bank in Atlanta, said most of her team performed well during the past year and were great about staying in touch with her. But the few who were not exhausted her.

“I would call a person during business hours, expecting them to answer the phone and might not get a callback for several hours,” Smith said. “Sometimes, they were quote ‘busy’ or had to run an errand, and sometimes they didn’t have any excuse.”

Smith just retired from her job because she was working seven days a week and had no work-life balance, she said. But she believes her team will do well with the hybrid mode, since up to 70% of their work can be done remotely.

And she recommends that any manager get buy-in from all team members regarding how the new system will work and to make sure when they ask people to come into the office that there’s a good reason for them to be there, such as for training or to boost their productivity if need be.

It’s more work for the manager

Dennis Meyer, a freelance television producer in Los Angeles who line-produces and manages budgets for shows like the Grammy Awards sees value in working from home, especially when the specific task doesn’t require someone to be on site.

But having everyone work from home on some days as a matter of course can mean he ends up doing things he’d normally delegate.

“You’re always weighing, ‘Isn’t it just faster if I do it myself?’” Meyer said. “Nothing is faster or more practical than just looking across the room and talking to someone with your voice. The instant you have to pick up a phone or write a message you’re wasting valuable time.”That’s especially true for him when the airing or taping of a show nears and he has to make a lot of decisions and changes. “You want someone nearby,” he said.

Dennis Meyer, a freelance television producer in Los Angeles, shown at his work-from-home setup.

Hybrid has to be better than all-remote

As a manager of a call center for an education company, Diana Garrett would prefer a hybrid situation to the all-remote one she is in now.

Garrett had only been in her job three months and was working at the Arizona office full-time when the pandemic struck.

Consequently, she’s only met 12 people in-person from her current team of 25. Everyone else has been hired since and they no longer are all based in the same state because the Arizona office was closed.

The lack of personal interactions makes her job harder and more time consuming, she said. “My tasks have remained the same, but it all takes so much more time. What could have been a 5-minute conversation in the office now takes double the time, if not more.”

And being on site with her team would make it easier for her to know exactly when they need support from her.

“A lot of my time is now spent discovering things I may or may not know. I would prefer some in face interaction. I find myself a lot more stressed.”

Just as productive working from home

Rene Cortes manages a sales and support team of about a dozen people for a medical device maker.

Cortes said his own work-from-home experience this past year has been enlightening. He used to fear that people would assume he wasn’t as productive working remotely.

But he discovered that was unfounded. “I’m just as productive as before and saved a lot of commuter time that I can use to support my customers and reps,” Cortes said.

Before the pandemic, he used to fly for work a lot, and take business calls in airports, he noted. “Now I have a quiet space at home when someone calls. I’m able to be more effective and not be distracted.”

Still, he added, “I missed the personal interaction at the office and interaction with customers.”

That’s been remedied by the hybrid schedule Cortes said his team has been on for months. They work in person during the middle of the week and work from home at least on Mondays and Fridays.

It’s been a very positive experience so far, he said. “The hybrid model appears to be the best approach going forward. It gives my team independence, but allows us to work together in person during key times while allowing me to be a more effective and efficient manager.”

Misses getting in a conference room together

Tracy Schumacher runs a team of 11 software developers and test engineers for a utility industry service provider in Syracuse, New York.

They worked on site full-time before the pandemic and have been all-remote ever since. But she’s currently polling team members to see which three days a week they want to work from home starting in mid-July.

Their experience as an all-remote team was mostly successful, but Schumacher felt that it was harder to onboard the three or four new employees she hired during the pandemic and to develop junior staff members. Plus, she added, on group calls it was harder to see who wanted to talk, so usually half the group remained silent.

She’s optimistic that the hybrid model will help remedy those issues.

“I think the thing we’ve missed is getting in a conference room together and drawing out solutions on a whiteboard,” Schumacher said.

That and getting more involvement from junior employees.

But extra care will be needed to develop them since the team will still be home much of the week. “Junior employees may not be as comfortable reaching out and asking questions. I think that’s where the challenge lies.”

She expects once everyone gets into a routine with their hybrid schedule that days in the office will feel more productive than the few days she’s spent there recently. So much of the time, she said, felt like it was spent catching up with people she hasn’t seen for ages.

Correction: The original version of this article misidentified which award show Dennis Meyer worked on. He has produced the Grammy Awards.