Had the same issue with my e36 it was the tie rod bushing. It will destroy your tyres in notime so get it sorted asap. I learned the hard way.