No, the Public Stream is the default stream where all your users (except for External) are invited, so it cannot be deleted. Use the Public Stream for company announcements or news.

If you don’t want users to participate in the Public Stream, you can just set them as “External”.