Website speed matters, it is a ranking factor. User tests have shown that slow loading sites cause users to browse away. There are many reasons to want a fast website, don't let those cheap hosts downplay this factor for your business!
We've written about this and given examples and shown results on why this is such a big deal. Read more here. I wanted to take this a step further and show you the exact verbiage from the Official Google Blog post :
"Here at Google we're obsessed with speed, in our products and on the web. As part of that effort, today we're including a new signal in our search ranking algorithms: site speed. Site speed reflects how quickly a website responds to web requests."