No it's very normal to take hours to draw sketches especially if you have no references. There was a post on Reddit about it the other day, even if you have a 'clear' picture of what you want to draw in your mind in actuality it is quite bare bones and time will be taken in order for you to decide how exactly those lines will be placed onto the paper.
Eventually you do get quicker, especially if you've drawn those folds for the billionth time and have rehashed a similar pose another billion times; things will just flow onto the page.
There's also of course the impulse to redraw lines and change your mind on how certain things will look.