Other, more obvious, things to leave out are things like tin-cans, rubber-bands and uranium. Paper and egg-shell cartons are probably ok. Teabags and coffee filters also work.
Oh, and watch out for pets, especially if you're starting your compost heap inside, Cats, dogs, bunnies and ferrets might think your pile 'o joy is their high-class commode. Too much ammonia isn't good for indoor worms, and cat shit carries an unfun disease called toxoplasma gondii which often shows no symptoms but can give developing fetuses brain-damage. Don't let cats play in your heap!