In , we introduced the . We learned about the type and the basic combinators like and . We saw how we could combine those together with applicative functions like and to parse strings into data structures. This week, we’ll put these pieces together in an actual parser for our Gherkin syntax. To follow along with the code examples, check out on the Github repository. last week’s article Applicative parsing library RE sym string many <*> Parser.hs Starting next week, we’ll explore some other parsing libraries, starting with . For a little more information about those and many other libraries, download our It summarizes many libraries on topics from databases to Web APIs. Attoparsec Production Checklist! If you’ve never written Haskell at all, get started! Download our free . Beginners Checklist! Value Parser In keeping with our approach from the last article, we’re going to start with smaller elements of our syntax. Then we can use these to build larger ones with ease. To that end, let’s build a parser for our type, the most basic data structure in our syntax. Let’s recall what that looks like: Value data Value =ValueNull |ValueBool Bool |ValueString String |ValueNumber Scientific Since we have different constructors, we’ll make a parser for each one. Then we can combine them with alternative syntax: valueParser :: RE Char ValuevalueParser = nullParser <|> boolParser <|> numberParser <|> stringParser Now our parsers for the null values and boolean values are easy. For each of them, we’ll give a few different options about what strings we can use to represent those elements. Then, as with the larger parser, we’ll combine them with . <|> nullParser :: RE Char ValuenullParser = (string “null” <|> string “NULL” <|> string “Null”) *> pure ValueNull boolParser :: RE Char ValueboolParser = trueParser *> pure (ValueBool True) <|> falseParser *> pure (ValueBool False) where trueParser = string “True” <|> string “true” <|> string “TRUE” falseParser = string “False” <|> string “false” <|> string “FALSE” Notice in both these cases we discard the actual string with and then return our constructor. We have to wrap the desired result with . *> pure Number and String Values Numbers and strings are a little more complicated since we can’t rely on hard-coded formats. In the case of numbers, we’ll account for integers, decimals, and negative numbers. We’ll ignore scientific notation for now. An integer is simple to parse, since we’ll have many characters that are all numbers. We use instead of to enforce that there is at least one: some many numberParser :: RE Char ValuenumberPaser = … where integerParser = some (psym isNumber) A decimal parser will read some numbers, then a decimal point, and then more numbers. We’ll insist there is at least one number after the decimal point. numberParser :: RE Char ValuenumberPaser = … where integerParser = some (psym isNumber) decimalParser = many (psym isNumber) <*> sym ‘.’ <*> some (psym isNumber) Finally, for negative numbers, we’ll read a negative symbol and then one of the other parsers: numberParser :: RE Char ValuenumberPaser = … where integerParser = some (psym isNumber) decimalParser = many (psym isNumber) <*> sym ‘.’ <*> some (psym isNumber) negativeParser = sym ‘-’ <*> (decimalParser <|> integerParser) However, we can’t combine these parsers as is! Right now, they all return different results! The integer parser returns a single string. The decimal parser returns two strings and the decimal character, and so on. In general, we’ll want to combine each parser’s results into a single string and then pass them to the function. This requires mapping a couple functions over our last two parsers: read numberParser :: RE Char ValuenumberPaser = … where integerParser = some (psym isNumber) decimalParser = combineDecimal <$> many (psym isNumber) <*> sym ‘.’ <*> some (psym isNumber) negativeParser = (:) <$> sym ‘-’ <*> (decimalParser <|> integerParser) combineDecimal :: String -> Char -> String -> String combineDecimal base point decimal = base ++ (point : decimal) Now all our number parsers return strings, so we can safely combine them. We’ll map the constructor over the value we read from the string. ValueNumber numberParser :: RE Char ValuenumberPaser = (ValueNumber . read) <$> (negativeParser <|> decimalParser <|> integerParser) where ... Note that order matters! If we put the integer parser first, we’ll be in trouble! If we encounter a decimal, the integer parser will greedily succeed and parse everything before the decimal point. We’ll either lose all the information after the decimal, or worse, have a parse failure. The last thing we need to do is read a string. We need to read everything in the example cell until we hit a vertical bar, but then ignore any whitespace. Luckily, we have the right combinator for this, and we’ve even written a function already! trim stringParser :: RE Char ValuestringParser = (ValueString . trim) <$> readUntilBar And now our will work as expected! valueParser Building an Example Table Now that we can parse individual values, let’s figure out how to parse the full example table. We can use our individual value parser to parse a whole line of values! The first step is to read the vertical bar at the start of the line. exampleLineParser :: RE Char [Value]exampleLineParser = sym ‘|’ *> ... Next, we’ll build a parser for each cell. It will read the whitespace, then the value, and then read up through the next bar. exampleLineParser :: RE Char [Value]exampleLineParser = sym ‘|’ *> ... where cellParser = many isNonNewlineSpace *> valueParser <* readThroughBar isNonNewlineSpace :: RE Char CharisNonNewlineSpace = psym (\c -> isSpace c && c /= ‘\n’) Now we read of these and finish by reading the newline: many exampleLineParser :: RE Char [Value]exampleLineParser = sym ‘|’ *> many cellParser <* readThroughEndOfLine where cellParser = many isNonNewlineSpace *> valueParser <* readThroughBar Now, we need a similar parser that reads the title column of our examples. This will have the same structure as the value cells, only it will read normal alphabetic strings instead of values. exampleColumnTitleLineParser :: RE Char [String]exampleColumnTitleLineParser = sym ‘|’ *> many cellParser <* readThroughEndOfLine where cellParser = many isNonNewlineSpace *> many (psym isAlpha) <* readThroughBar Now we can start building the full example parser. We’ll want to read the string, the column titles, and then the value lines. exampleTableParser :: RE Char ExampleTableexampleTableParser = (string “Examples:” *> readThroughEndOfLine) *> exampleColumnTitleLineParser <*> many exampleLineParser We’re not quite done yet. We’ll need to apply a function over these results that will produce the final . And the trick is that we want to map up the example keys with their values. We can accomplish this with a simple function. It will return zip the keys over each value list using : ExampleTable map exampleTableParser :: RE Char ExampleTableexampleTableParser = buildExampleTable <$> (string “Examples:” *> readThroughEndOfLine) *> exampleColumnTitleLineParser <*> many exampleLineParser where buildExampleTable :: [String] -> [[Value]] -> ExampleTable buildExampleTable keys valueLists = ExampleTable keys (map (zip keys) valueLists) Statements Now we that we can parse the examples for a given scenario, we need to parse the Gherkin statements. To start with, let’s make a generic parser that takes the keyword as an argument. Then our full parser will try each of the different statement keywords: parseStatementLine :: String -> RE Char StatementparseStatementLine signal = … parseStatement :: RE Char StatementparseStatement = parseStatementLine “Given” <|> parseStatementLine “When” <|> parseStatementLine “Then” <|> parseStatementLine “And” Now we’ll get the signal word out of the way and parse the statement line itself. parseStatementLine :: String -> RE Char StatementparseStatementLine signal = string signal *> sym ' ' *> ... Parsing the statement is tricky. We want to parse the keys inside brackets and separate them as keys. But we also want them as part of the statement’s string. To that end, we’ll make two helper parsers. First, will parse everything in a string up through a bracket (or a newline). nonBrackets nonBrackets :: RE Char StringnonBrackets = many (psym (\c -> c /= ‘\n’ && c /= ‘<’)) We’ll also want a parser that parses the brackets and returns the keyword inside: insideBrackets :: RE Char StringinsideBrackets = sym ‘<’ *> many (psym (/= ‘>’)) <* sym ‘>’ Now to read a statement, we start with non-brackets, and alternate with keys in brackets. Let’s observe that we start and end with non-brackets, since they can be empty. Thus we can represent a line a list of non-bracket/bracket pairs, followed by a last non-bracket part. To make a pair, we combine the parser results in a tuple using the constructor enabled by : (,) TupleSections parseStatementLine :: String -> RE Char StatementparseStatementLine signal = string signal *> sym ‘ ‘ *> many ((,) <$> nonBrackets <*> insideBrackets) <*> nonBrackets From here, we need a recursive function that will build up our final statement string and the list of keys. We do this with . buildStatement parseStatementLine :: String -> RE Char StatementparseStatementLine signal = string signal *> sym ‘ ‘ *> (buildStatement <$> many ((,) <$> nonBrackets <*> insideBrackets) <*> nonBrackets) where buildStatement :: [(String, String)] -> String -> (String, [String]) buildStatement [] last = (last, []) buildStatement ((str, key) : rest) rem = let (str', keys) = buildStatement rest rem in (str <> "<" <> key <> ">" <> str', key : keys) The last thing we need is a final helper that will take the result of and turn it into a . We’ll call this , and then we’re done! buildStatement Statement finalizeStatement parseStatementLine :: String -> RE Char StatementparseStatementLine signal = string signal *> sym ‘ ‘ *> (finalizeStatement . buildStatement <$> many ((,) <$> nonBrackets <*> insideBrackets) <*> nonBrackets) where buildStatement :: [(String, String)] -> String -> (String, [String]) buildStatement [] last = (last, []) buildStatement ((str, key) : rest) rem = let (str', keys) = buildStatement rest rem in (str <> "<" <> key <> ">" <> str', key : keys) finalizeStatement :: (String, [String]) -> Statement finalizeStatement (regex, variables) = Statement regex variables Scenarios Now that we have all our pieces in place, it’s quite easy to write the parser for scenario! First we get the title by reading the keyword and then the rest of the line: scenarioParser :: RE Char ScenarioscenarioParser = string “Scenario: “ *> readThroughEndOfLine ... After that, we read many statements, and then the example table. Since the example table might not exist, we’ll provide an alternative that is a pure, empty table. We can wrap everything together by mapping the constructor over it. Scenario scenarioParser :: RE Char ScenarioscenarioParser = Scenario <$> (string “Scenario: “ *> readThroughEndOfLine) <*> many (statementParser <* sym ‘\n’) <*> (exampleTableParser <|> pure (ExampleTable [] [])) We can also make a “Background” parser that is very similar. All that changes is that we read the string “Background” instead of a title. Since we’ll hard-code the title as “Background”, we can include it with the constructor and map it over the parser. backgroundParser :: RE Char ScenariobackgroundParser = Scenario “Background” <$> (string “Background:” *> readThroughEndOfLine) *> many (statementParser <* sym ‘\n’) <*> (exampleTableParser <|> pure (ExampleTable [] [])) Finally the Feature We’re almost done! All we have left is to write the itself! As with scenarios, we’ll start with the keyword and a title line: featureParser featureParser :: RE Char FeaturefeatureParser = Feature <$> (string “Feature: “ *> readThroughEndOfLine) <*> ... Now we’ll use the combinator to parse the if it exists, but return if it doesn’t. Then we’ll wrap up with parsing many scenarios! optional Background Nothing featureParser :: RE Char FeaturefeatureParser = Feature <$> (string “Feature: “ *> readThroughEndOfLine) <*> (optional backgroundParser) <*> (many scenarioParser) Note that here we’re ignoring the “description” of a feature we proposed as part of our original syntax. Since there are no keywords for that, it turns out to be painful to deal with it using applicative parsing. When we look at monadic approaches starting next week, we’ll see it isn’t as hard there. Conclusion This wraps up our exploration of applicative parsing. We can see how well suited Haskell is for parsing. The functional nature of the language means it’s easy to start with small building blocks like our first parsers. Then we can gradually combine them to make something larger. It can be a little tricky to wrap our heads around all the different operators and combinators. But once you understand the ways in which these let us combine our parsers, they make a lot of sense and are easy to use. To further your knowledge of useful Haskell libraries, download our free ! It will tell you about libraries for many tasks, from databases to machine learning! Production Checklist If you’ve never written a line of Haskell before, never fear! Download our to learn more! Beginners Checklist
