greg.blog

Tag: ElasticSearch

  • Six Use Cases of Elasticsearch in WordPress

    I spoke at WordCamp US two weeks ago about six different use cases I’ve seen for Elasticsearch within the WordPress community. I also mentioned two projects for WordPress.org that are planned for 2016 and involve Elasticsearch if anyone is looking for opportunities to learn to use Elasticsearch and contribute to WordPress: WordPress.tv is getting rebuilt…

  • Elasticsearch: The Broken Bits

    A post like this carries a heavy risk of this: Translation: “Your time is far less valuable than mine, open source slave.” pic.twitter.com/D1AkwxG5qg — Ryan Grove (@yaypie) August 5, 2014 So, some friendly caveats: I wrote the first draft of this a bit after midnight in the middle of a week where we weathered 3…

  • Scaling Elasticsearch Part 3: Queries

    See part 1 and part 2 for an overview of our system and how we scale our indexing. Originally I was planning a separate post for global queries and related posts queries, but it was hard to break into two posts and contributed to me taking forever to write them. Two types of queries run…

  • Presentation: Elasticsearch at Automattic

    I gave a presentation at the Elasticsearch Denver meetup last night covering some of the bigger changes we had to make to scale a cluster to handle all WordPress.com posts. This is a cherry picked list from my more comprehensive ongoing series of posts chronicling our experience scaling Elasticsearch. Presentation slides don’t always translate well…

  • Elasticsearch, Open Source, and the Future

    This essay started as a response to a comment on my multilingual indexing post. The comment is mostly an advertisement, but brings up some interesting points so I decided to publish it and turn my response into a full post. For some context here’s the key part of the comment: I thought readers might be…

  • Scaling Elasticsearch Part 2: Indexing

    In part 1 I gave an overview of our cluster configuration. In this part we’ll dig into: How our data is partitioned into indices to scale over time Optimizing bulk indexing Scaling real time indexing How we manage indexing failures and downtime. The details of our document mappings are mostly irrelevant for our indexing scaling…

  • Scaling Elasticsearch Part 1: Overview

    We recently launched Related Posts across WordPress.com, so its time to pop the hood and take a look at what ended up in our engine. There’s a lot of good information spread across the web on how to use Elasticsearch, but I haven’t seen too many detailed discussions of what it looks like to scale…

  • Managing Elasticsearch Cluster Restart Time

    While building a fairly large index (8TB total for 500 million docs), I ran into some very long restart times for the cluster. That prompted me to start a discussion about long restart times. There’s some good discussion in that thread, and I wanted to write a post to summarize what we are doing to…

  • Introducing Related Posts

    Do you ever wonder what happens when your readers reach the end of your posts? What do they click on? Where do they go next? What if you’ve piqued a reader’s interest and left them wanting more, but don’t give them the option to do so? Today, we’re so happy to announce Related Posts on…

  • Three Principles for Multilingal Indexing in Elasticsearch

    Recently I’ve been working on how to build Elasticsearch indices for WordPress blogs in a way that will work across multiple languages. Elasticsearch has a lot of built in support for different languages, but there are a number of configuration options to wade through and there are a few plugins that improve on the built…