I use vermiculite often, it's in every potted plant. So look out its everywhere. Basically harmless, could have a remote possibly of have a trace amount of asbestos. Higher concentrations in more of your household items than in vermiculite. It's natural and it won't hurt to throw away.
Great high temp insulation for outdoor ovens and small back yard forges.