So I tagged my shows like: Currently Watching, Stopped Watching, Finished, Waiting List etc. On this site under Calendar I choose the Currently Watching tag obviously. But the iCal shows every followed shows, shows that I don't watch anymore/yet. There should be an option to choose which tag's shows would be listed in the iCal.
My workaround is, since the tags are still there after unfollowing, is "simply" unfollow the shows that I don't currently watch.