The business owner said these three issues make Portland "easily the worst market in the country" to operate a restaurant.
Oregon Gov. Kate Brown recently announced she is moving 15 of Oregon's biggest counties into the "extreme risk" category due to rising COVID-19 infections and hospitalizations, forcing another shutdown of indoor dining for restaurants that have been battered by pandemic restrictions over the last 13 months.
Additionally, Portland has contended with months of violent demonstrations and clashes since George Floyd’s death last May. In late April, a group of about 80 people caused nearly $20,000 worth of property damage to a Boys and Girls Club in northeast Portland.
"It's devastated the image and the reputation of Portland," Huffman told host Stuart Varney. "A third of our dining population in downtown Portland are tourists. So when it kills that, it really hurts us."
Huffman has been forced to permanently close two of his restaurants.
"We've actually fared better, I think, than anyone here in this town," he said. "The closure rates here are amazing."
Huffman added he worries about smaller, independent restaurants that don’t have the resources to survive.
Huffman said his business will be focusing growth efforts "elsewhere" but despite all the hardship, he hasn’t considered leaving his hometown of Portland.
"I'm loyal to the cause. It's just super frustrating because it's such an awesome town," he said. "And right now, reputationally, it's just embarrassing."
FOX Business’ Thomas Barrabi and Paul Best contributed to this article.