Evicted two days after the lease was sign because boyfriend is a smoker, he didn't smoke near the apartment?

So, my boyfriend and myself paid the first and last months rent along with a security deposit and then sign a lease on an apartment which at first seemed so perfect. We began moving our things in but decided we weren't going to move in until our furniture arrived. In the lease it clearly states that there is no smoking IN the apartment. Today we arrived at our apartment to find a note on the door saying " Hilary and Rod, don't move anything else into my apartment. We need to talk. I'll be back around 7 tonite." (yes he spelled tonight incorrectly) Needless to say we showed up at the apartment 15 minutes early thinking that our landlord needed to speak with us about the hardwood floors or something never to realize that he was going to evict us on our third day of being his tenants. We waited in our car until he arrived and he rudely said that he wants our stuff out of his apartment by tomorrow (the day we are leaving for vacation) and he will reimburse us our money. His "reasoning" is that someone he knows saw Rodney (my boyfriend) smoking. Not on his property but around town. Rodney has been trying to quit smoking for a very long time mainly because i am seven months pregnant and I don't want our child to be raised thinking it's okay to smoke and also because he's tired of having to pay for cigarettes, smelling like them, and that horrible smokers cough. Rodney rarely smokes anymore, only when he needs that little bit of nicotine to keep him going but he never smokes around me anymore because he's actually putting effort into quiting. We did some research and from what we gathered we can't be convicted unless we broke the lease or didn't pay the rent, but the thing is we didn't break the lease at all and we are paying multiple months of rent in advanced! The landlord claims that he has the right to just rip up the lease and have that be the end of it, but we are hoping we can take legal action. We just spent almost $3,000 on stuff to match that apartment and now what are we suppose to do with it all since we have to move back in with his parents. What i am trying to ask is, does he have a legitimate reason to evict us? He claims he has 3 days to change his mind on the lease but it says nowhere on the internet if he is telling the truth. Please help. We want to know if it's even worth taking legal action.