Are polynomial features the root of all evil? A myth