Michele Riva

Writing a full-text search engine in TypeScript

Can TypeScript power a search engine that queries a million titles in 41 microseconds? Discover the data structures and algorithms that make it possible.

Writing a full-text search engine in TypeScript
#1about 2 minutes

Why build a full-text search engine from scratch

Building a search engine from scratch is the best way to understand the underlying data structures and algorithms that power it.

#2about 2 minutes

An overview of existing full-text search solutions

Full-text search uses text indexes to quickly find terms, with established solutions like Elasticsearch, Algolia, and newer ones like Meilisearch.

#3about 6 minutes

Preparing text data with tokenization and stemming

Raw text is processed through tokenization, lowercasing, stop-word removal, and stemming to create a clean set of searchable terms.

#4about 6 minutes

Using hash maps to create an inverted index

An inverted index, implemented with a hash map, provides constant-time (O(1)) lookups by mapping search tokens directly to the documents that contain them.

#5about 8 minutes

Optimizing storage space with prefix trees (tries)

Prefix trees, or tries, optimize memory usage by storing common prefixes of words only once, avoiding redundant data storage.

#6about 9 minutes

Implementing typo tolerance with Levenshtein distance

The Levenshtein distance algorithm uses dynamic programming to calculate the "edit distance" between two strings, enabling effective typo tolerance in search queries.

#7about 2 minutes

Introducing Lyra, a fast TypeScript search engine

Lyra is a new, open-source full-text search engine written in TypeScript that achieves microsecond search times by leveraging efficient data structures.

#8about 3 minutes

Q&A on hash functions and memory constraints

The Q&A covers the educational value of custom hash functions, handling acronyms versus stop words, and Lyra's current in-memory architecture.

Related jobs
Jobs that call for the skills explored in this talk.

Designer

Andrew Comp

Andrew Comp
Berlin, Germany

Intermediate
Java
JavaScript

Matching moments

Q&A on TypeScript, clean code, and algorithms
02:08 MIN

Q&A on TypeScript, clean code, and algorithms

Things I learned while writing high-performance JavaScript applications

Unlock full access

Log in or set up an account to access this feature and more.

The four pillars of high-performance JavaScript
04:53 MIN

The four pillars of high-performance JavaScript

Things I learned while writing high-performance JavaScript applications

Unlock full access

Log in or set up an account to access this feature and more.

Q&A on performance, interfaces, and advanced learning
04:23 MIN

Q&A on performance, interfaces, and advanced learning

Where we're going we don't need JavaScript - Programming with Type Annotations

Unlock full access

Log in or set up an account to access this feature and more.

Q&A on indexing, aggregations, and OpenSearch vs Elasticsearch
03:31 MIN

Q&A on indexing, aggregations, and OpenSearch vs Elasticsearch

Search and aggregations made easy with OpenSearch and NodeJS

Unlock full access

Log in or set up an account to access this feature and more.

Introducing the core principles of Elasticsearch
04:29 MIN

Introducing the core principles of Elasticsearch

Distributed search under the hood

Unlock full access

Log in or set up an account to access this feature and more.

Understanding TypeScript's origins and role in scalability
02:37 MIN

Understanding TypeScript's origins and role in scalability

All you need is types

Unlock full access

Log in or set up an account to access this feature and more.

Audience Q&A on patterns, typescript, and frameworks
02:59 MIN

Audience Q&A on patterns, typescript, and frameworks

10 must-know design patterns for JS Devs

Unlock full access

Log in or set up an account to access this feature and more.

Exploring the key benefits of adopting TypeScript
01:56 MIN

Exploring the key benefits of adopting TypeScript

Do TypeScript without TypeScript

Unlock full access

Log in or set up an account to access this feature and more.

Featured Partners

Related Videos

See all videos
Things I learned while writing high-performance JavaScript applications27:27

Things I learned while writing high-performance JavaScript applications

Michele Riva

Building software that scales with Typescript37:18

Building software that scales with Typescript

Tal Joffe

Advanced Typing in TypeScript28:44

Advanced Typing in TypeScript

Lars Hupel

Lies we Tell Ourselves As Developers31:13

Lies we Tell Ourselves As Developers

Stefan Baumgartner

Don't compromise on speedy delivery nor type-safety by choosing TypeScript29:44

Don't compromise on speedy delivery nor type-safety by choosing TypeScript

Jens Claes

Where we're going we don't need JavaScript - Programming with Type Annotations29:30

Where we're going we don't need JavaScript - Programming with Type Annotations

Peter Kröner

All you need is types48:56

All you need is types

Tal Joffe

Langium: Design Your Own Language in Node.js and VS Code45:59

Langium: Design Your Own Language in Node.js and VS Code

Miro Spönemann

Related Articles

View all articles
CH
Chris Heilmann
All the videos of Halfstack London 2024!
Last month was Halfstack London, a conference about the web, JavaScript and half a dozen other things. We were there to deliver a talk, but also to record all the sessions and we're happy to share them with you. It took a bit as we had to wait for th...
All the videos of Halfstack London 2024!
DC
Daniel Cranney
Stephan Gillich - Bringing AI Everywhere
In the ever-evolving world of technology, AI continues to be the frontier for innovation and transformation. Stephan Gillich, from the AI Center of Excellence at Intel, dove into the subject in a recent session titled "Bringing AI Everywhere," sheddi...
Stephan Gillich - Bringing AI Everywhere
LM
Luis Minvielle
The Best Upcoming IT Webinars
Now that you already know what IT webinars are and how they can help you level up your professional appeal, you might want actually to get into one. Live tech webinars are one of the best ways to stay on top of the latest trends and tools because eit...
The Best Upcoming IT Webinars

From learning to earning

Jobs that call for the skills explored in this talk.