The main fly in the honey is the city requirement that all vendors make their products in commercial kitchens, not at home — which is sort of the whole point of the market.
The vendors don't want to go through the permitting process because it is too expensive. The Greenmarket allows the vendors to skirt the standard business model--including costs--similarly to how cheap curb parking benefits food trucks.
Should businesses be allowed to operate quasi-informally like this? Maybe. New York is really expensive and perhaps food trucks and Greenmarkets give entrepreneurs a chance to try out ideas with relatively low risk. That's good, especially if they graduate to a formal operation. If rents and permits are significant barriers to entry they need to be reconsidered. But it's not clear that rents and permits are barriers to entry. Assuming a relatively inelastic market for these food items businesses that operate quasi-informally are likely taking business from those that pay high rents for market access and follow the permitting process for health.