Low back pain is the leading cause of years lived with disability globally. In 2018, an international working group called on the World Health Organization to increase attention on the burden of low back pain and the need to avoid excessively medical solutions. Indeed, major international clinical guidelines now recognize that many people with low back pain require little or no formal treatment. Where treatment is required the recommended approach is to discourage use of pain medication, steroid injections and spinal surgery, and instead promote physical and psychological therapies. Many health systems are not designed to support this approach. In this paper we discuss why care for low back pain that is concordant with guidelines requires system-wide changes. We detail the key challenges of low back pain care within health systems. These include the financial interests of pharmaceutical and other companies; outdated payment systems that favour medical care over patients’ self-management; and deep-rooted medical traditions and beliefs about care for back pain among physicians and the public. We give international examples of promising solutions and policies and practices for health systems facing an increasing burden of ineffective care for low back pain. We suggest policies that, by shifting resources from unnecessary care to guideline-concordant care for low back pain, could be cost-neutral and have widespread impact. Small adjustments to health policy will not work in isolation, however. Workplace systems, legal frameworks, personal beliefs, politics and the overall societal context in which we experience health, will also need to change.