A floppy disk, USB keychain, and any other small(ish) storage device often comes as a blank slate or formatted for Windows or Macintosh use (most often these days they come formatted for Windows). If the item is a blank slate, no computer can use it for anything. Many come by default formatted for Windows, which you can use in Linux with no problem — or you can change its formatting into a Linux setup. (The handy thing about leaving it as a Windows disk is that you can then use it to share things with Windows users.)
To format a floppy, place the floppy into your floppy drive and choose PlacesComputer. In the Computer dialog box, right-click the Floppy Drive icon and choose Format from the context menu.
Typically, the defaults are what you want to choose. If you want to format the disks for Linux, use the Linux Native (ext2) format. To share disks with Windows users, choose DOS (FAT). Another setting you may want to choose is Thorough rather than Quick.
This same technique works for many types of removable media. Locate the appropriate icon in the Computer dialog box and right-click it just as you would for a floppy. Many devices like USB thumb drives are sold pre-formatted for DOS (FAT). It is fine to leave them in this form.