Hey I'd just ignore their threats, often companies that feel threatened try to throw their weight around. But as long as you have no contract with your past employer that you have no legal obligation to not post your work up wherever you like.
I think it only really gets to be an issue if you were to sell on anything you did whilst being paid by them or were to give away any trade secrets or individuals identities.
If it's of great concern I'd suggest sending them off a polite letter, turning down the offer of withdrawing your work from your portfolio and asking them for a copy of a document that concludes that they are right in pursuing legal action against you. If you're feeling brave that is ;D
