Most people who are sugar conscious these days are really carb conscious.
Sugar is but one carb and assuming you are talking about a bread product
(you did add yeast) it is chock full of carbs more so from the flour than
the sugar. All carbs convert to glucose in your body and glucose is a form
of sugar.

If the problem is simply a desire not to eat table sugar (sucrose) then yes
there is still a negligible amount in the product. Sugar acts as a catalyst
for yeast but usually does not convert it all in a recipe.