For almost 80 years, Salem remained the center of the spice trade for the new world. Fortunes were made, ships were built, and Salem became for a time the 6th largest city in the U.S., the richest city in the U.S., and had the first millionaire in the U.S. (Hence the Derby St. running near the waterfront which bears the name of that first mega-fortune individual William H. Derby.)
Salem was, during the early 1800's, both a magnificent place to see the wonders of the far east trade, and a jumping off place for adventure. As ships returned from their voyages, they would tie up at one of the many piers built into the harbor. Piers would stretch for thousands of feet into the water and hundreds of ships at a time would be in port. Languages and goods from all over the world could be found in Salem. it was said that the King of Sumatra one day looked over his own harbor and stated to those present that "Salem must be a wondrously rich country to have so many ships based there." To this day, the Salem city seal depicts a man in traditional Sumatran garb.