Learning from context is harder than we thought