Website speed matters, it is a ranking factor.  User tests have shown that slow loading sites cause users to browse away.  There are many reasons to want a fast website, don't let those cheap hosts downplay this factor for your business!

We've written about this and given examples and shown results on why this is such a big deal.  Read more here.  I wanted to take this a step further and show you the exact verbiage from the Official Google Blog post :

"Here at Google we're obsessed with speed, in our products and on the web. As part of that effort, today we're including a new signal in our search ranking algorithms: site speed. Site speed reflects how quickly a website responds to web requests."