So you want better OpenSearch documentation?

Charlie Hull and I, Eric Pugh, from OpenSource Connections, had the opportunity during the “Unconference” portion of OpenSearchCon Europe 2024 to talk about the current state of the OpenSearch documentation and get some feedback from the community. I chose this topic based on conversations I had with Heather Halter, OpenSearch Documentation Manager, last fall at OpenSearchCon 2023 in Seattle.

We started with the slightly tongue-in-cheek statement that the way you search Reddit is to go to Google and search “How do I (insert any task here) +Reddit” to get the information you want. The pattern that lots of folks use to find information about OpenSearch (unsurprisingly, as OpenSearch started as a fork of Elasticsearch) is to go to Google and search “How do I (insert any task here) +Elasticsearch.” That statement was met with a few nodding heads and light chuckling.

We then introduced The Documentation System, which is referred to as The Grand Unified Theory of Documentation. You may know it as Diátaxis (I am unclear about the relationship between the two sites!). Here’s a video explaining the concept, which divides documentation into four quadrants: Tutorials, How-To Guides, Explanation, and Reference.

OpenSearch actually does a pretty good job of the Reference portion of the puzzle, especially considering that there are new releases every 6 weeks. I started contributing documentation late last year and recently joined the opensearch-project/documentation-website repository as a maintainer. I see a constant flow of both new docs and the improvement of existing docs. Considering the ratio of writers to developers and where the project was not too long ago, I think we’re doing a really good job of keeping up with both.

The other three quadrants, however, are where we could do a better job. There is Tutorial and Explanation content mixed in with the Reference content, so you can spend a lot of time wading through the minutiae of how things work when you are really looking for an understanding of the core principles. We also have a huge amount of great content in the existing blog that could flesh out these other areas of the documentation, especially if we start linking to third-party blogs. This is an area I plan to invest more energy into.

Charlie then asked the audience what their thoughts were, and we captured a lot of great questions and comments. I then followed up with Heather Halter, whose remarks are in italics.

“Older versions of OpenSearch docs often show up in Google, which is often jarring. Can we make sure Google is properly indexing the latest content?”

While we’ve made a lot of progress in this area, particularly with implementing a redirect strategy, we are currently planning on adding canonical links to all the older pages. The goal is to get this work done by the end of Q2, which should show a big improvement in the back half of the year.

“I often find a GitHub issue that answers my question and I ⭐ it.” Someone suggested that we look for comments on GitHub issues that have lots of stars and use that as an indicator of where docs could be improved.

Great suggestion!

“My 2-year-old won’t let me work at night” was one response to the question of why people who find a gap in documentation don’t pursue opening a PR to improve the docs.

We love community contributions and have made it really easy to start. Check out the CONTRIBUTING.md file and watch this YouTube video on our documentation process and how you can contribute. You can also search through the issues labeled good first issue or help wanted.

Backwards compatibility in APIs came up as an issue. It would be interesting if we could better communicate API changes over time.

We use the “Introduced (version)” label to indicate the release in which a feature is introduced. But going backwards to identify everything is a big job. We received a suggestion to do this for field types, so I labeled it with “good first issue” in case someone wants to jump in.

Someone did make the comment that they do NOT believe that developers don’t like to write docs, especially in open source, but that we need to strive to continue to lower the barriers to contributing docs.

We are relying more and more on developers to write content and are looking at tools that make it easier and faster to write good documentation. We also have a great automated style checker built into our process that helps developers learn how to write better by correcting their mistakes.

One person asked what powered the site search functionality, and the response that it is OpenSearch sparked a conversation about whether we could apply new OpenSearch capabilities, like hybrid search and vectors, to what powers the website. Kris Freedain spoke up with a “Who wants to collaborate on this?”, and at least one person raised their hand ;-).

We are currently looking at how we can upgrade the search experience on the overall website. Please get in touch with Kris Freedain if you are interested.

If you have more feedback on OpenSearch documentation, reach out on the forum, Slack, or enter an issue in the documentation-website repo. We also welcome you to contribute your thoughts on the recently published OpenSearch Documentation Strategy.

Thanks to the OpenSearch team for providing a great forum to connect with an incredible community. You can catch up on all the excitement in this blog post, and I look forward to seeing everyone at OpenSearchCon North America!