Ruby is not only a fun language, but it also comes with an excellent standard library. Some of which are not that known, and are almost hidden Gems. Today we’ll pay attention to one of them. StringScanner Ruby gem provides lexical scanning operations on a string.
So what does “lexical scanning” mean exactly? Essentially it describes the process of taking an input string and extracting meaningful bits of information from it, following certain rules.
For example, this can be seen at the first stage of a compiler which takes an expression like
2 + 1 as input and turns it into the following sequence of tokens:
However, sometimes our parsing needs aren’t that elaborate, and a simpler library like the regular expression based
StringScanner can come in very handy in such situations. It works by remembering the location of a so-called scan pointer which is nothing more than an index into the string. The scanning process then tries to match the code right after the scan pointer with the provided expression. Apart from matching operations,
StringScanner also provides methods for moving the scan pointer (moving forwards or backwards through the string), looking ahead (seeing what’s next without modifying the scan pointer just yet) as well as finding out where in the string we currently are (is it the beginning or end of a line/the entire string etc).
StringScanner occupies a nice middle ground between simple regular expressions and a full-blown lexer. It isn’t the best choice for complex scanning and parsing needs. But it’s straightforward nature makes it easy for everyone with basic regular expression knowledge to extract information from input strings and I’ve used those successfully in production code in the past. We hope you’ll discover this hidden Gem.
Interested in more examples? Find them here.
And don’t miss the information about RubyGems that was updated due to some vulnerabilities, so we recommend you change it to the latest stable version.
Also check our latest Rails digest with the most interesting Github repositories in March!