Ecommerce site - listing attributes of unlisted items?

I am starting to set up my ecommerce site.

We are not listing all of our items up on the ecommerce site.

But, when we look at the display of the few test items that we added to the ecommerce, there is a complete list of every single attribute of all the items on the left hand side of the page, even the unlisted items.

We are starting off with a a dozen or so items form our catalog of almost 200 items.

Having all the unneeded attributes list causes confusion of the items that are unlisted.

I figured out that we can turn off the listing of full categories of attributes.

If I list a red shirt, in the attribute list, it will list: red, blue, yellow, black, green orange, etc…

It is listing all attributes, while what we want is only those of the items that are listed in the store.

How can this list be changed to list only the attributes of the items in the ecommerce?