Your assumption matches the facts. Most plants that rely on animals for reproduction are endangered or extinct without humans cultivating them. There's a school of thought positing that ancient humans cultivated them from scratch over thousands of years, but there's no pre-poké-religious tradition nor any archaeological record to back that up.