Crate nom [−] [src]

Nom, eating data byte by byte

The goal is to make a parser combinator library that is safe, supports streaming (push and pull), and as much as possible zero copy.

The code is available on Github

Example

#[macro_use]
extern crate nom;

use nom::{IResult,digit,multispace};
use nom::IResult::*;

// Parser definition

use std::str;
use std::str::FromStr;

named!(parens<i64>, delimited!(
    delimited!(opt!(multispace), tag!("("), opt!(multispace)),
    expr,
    delimited!(opt!(multispace), tag!(")"), opt!(multispace))
  )
);

// We transform an integer string into a i64
// we look for a digit suite, and try to convert it.
// if either str::from_utf8 or FromStr::from_str fail,
// the parser will fail
named!(factor<i64>,
  alt!(
    map_res!(
      map_res!(
        delimited!(opt!(multispace), digit, opt!(multispace)),
        str::from_utf8
      ),
      FromStr::from_str
    )
  | parens
  )
);

// we define acc as mutable to update its value whenever a new term is found
named!(term <i64>,
  chain!(
    mut acc: factor  ~
             many0!(
               alt!(
                 tap!(mul: preceded!(tag!("*"), factor) => acc = acc * mul) |
                 tap!(div: preceded!(tag!("/"), factor) => acc = acc / div)
               )
             ),
    || { return acc }
  )
);

named!(expr <i64>,
  chain!(
    mut acc: term  ~
             many0!(
               alt!(
                 tap!(add: preceded!(tag!("+"), term) => acc = acc + add) |
                 tap!(sub: preceded!(tag!("-"), term) => acc = acc - sub)
               )
             ),
    || { return acc }
  )
);

fn main() {
  assert_eq!(expr(&b" 1 +  2 "[..]),             IResult::Done(&b""[..], 3));
  assert_eq!(expr(&b" 12 + 6 - 4+  3"[..]),      IResult::Done(&b""[..], 17));
  assert_eq!(expr(&b" 1 + 2*3 + 4"[..]),         IResult::Done(&b""[..], 11));

  assert_eq!(expr(&b" (  2 )"[..]),              IResult::Done(&b""[..], 2));
  assert_eq!(expr(&b" 2* (  3 + 4 ) "[..]),      IResult::Done(&b""[..], 14));
  assert_eq!(expr(&b"  2*2 / ( 5 - 1) + 3"[..]), IResult::Done(&b""[..], 4));
}

Macros

add_error!	Add an error if the child parser fails
alt!	`alt!(I -> IResult<I,O> \| I -> IResult<I,O> \| ... \| I -> IResult<I,O> ) => I -> IResult<I, O>` try a list of parsers, return the result of the first successful one
apply!	emulate function currying: `apply!(my_function, arg1, arg2, ...)` becomes `my_function(input, arg1, arg2, ...)`
bits!	`bits!( parser ) => ( &[u8], (&[u8], usize) -> IResult<(&[u8], usize), T> ) -> IResult<&[u8], T>` transforms its byte slice input in a bit stream for the underlying parsers
call!	Used to wrap common expressions and function as macros
chain!	`chain!(I->IResult<I,A> ~ I->IResult<I,B> ~ ... I->IResult<I,X> , \|\| { return O } ) => I -> IResult<I, O>` chains parsers and assemble the results through a closure the input type I must implement nom::InputLength this combinator will count how much data is consumed by every child parser and take it into account if there is not enough data
char!	matches one character: `char!(char) => &[u8] -> IResult<&[u8], char>
closure!	Wraps a parser in a closure
complete!	replaces a `Incomplete` returned by the child parser with an `Error`
cond!	`cond!(bool, I -> IResult<I,O>) => I -> IResult<I, Option<O>>` Conditional combinator
cond_reduce!	`cond_reduce!(bool, I -> IResult<I,O>) => I -> IResult<I, O>` Conditional combinator with error
consumer_from_parser!
count!	`count!(I -> IResult<I,O>, nb) => I -> IResult<I, Vec<O>>` Applies the child parser a specified number of times
count_fixed!	`count_fixed!(O, I -> IResult<I,O>, nb) => I -> IResult<I, [O; nb]>` Applies the child parser a fixed number of times and returns a fixed size array The type must be specified and it must be `Copy`
dbg!	Prints a message if the parser fails
dbg_dmp!	Prints a message and the input if the parser fails
delimited!	`delimited!(I -> IResult<I,T>, I -> IResult<I,O>, I -> IResult<I,U>) => I -> IResult<I, O>` delimited(opening, X, closing) returns X
error!	Prevents backtracking if the child parser fails
escaped!
escaped_transform!
expr_opt!	`expr_opt!(Option<O>) => I -> IResult<I, O>` evaluate an expression that returns a Option and returns a IResult::Done(I,T) if Ok
expr_res!	`expr_res!(Result<E,O>) => I -> IResult<I, O>` evaluate an expression that returns a Result and returns a IResult::Done(I,T) if Ok
fix_error!	translate parser result from IResult to IResult woth a custom type
flat_map!	`flat_map!(R -> IResult<R,S>, S -> IResult<S,T>) => R -> IResult<R, T>`
i16!	if parameter is true, parse a big endian i16 integer, otherwise a little endian i16 integer
i32!	if parameter is true, parse a big endian i32 integer, otherwise a little endian i32 integer
i64!	if parameter is true, parse a big endian i64 integer, otherwise a little endian i64 integer
is_a!	`is_a!(&[T]) => &[T] -> IResult<&[T], &[T]>` returns the longest list of bytes that appear in the provided array
is_a_bytes!
is_not!	`is_not!(&[T:AsBytes]) => &[T] -> IResult<&[T], &[T]>` returns the longest list of bytes that do not appear in the provided array
is_not_bytes!
length_bytes!	`length_bytes!(&[T] -> IResult<&[T], nb>) => &[T] -> IResult<&[T], &[T]> gets a number from the first parser, then extracts that many bytes from the remaining stream
length_value!	`length_value!(I -> IResult<I, nb>, I -> IResult<I,O>) => I -> IResult<I, Vec<O>>` gets a number from the first parser, then applies the second parser that many times
many0!	`many0!(I -> IResult<I,O>) => I -> IResult<I, Vec<O>>` Applies the parser 0 or more times and returns the list of results in a Vec
many1!	`many1!(I -> IResult<I,O>) => I -> IResult<I, Vec<O>>` Applies the parser 1 or more times and returns the list of results in a Vec
map!	`map!(I -> IResult<I,O>, O -> P) => I -> IResult<I, P>` maps a function on the result of a parser
map_opt!	`map_opt!(I -> IResult<I,O>, O -> Option<P>) => I -> IResult<I, P>` maps a function returning an Option on the output of a parser
map_res!	`map_res!(I -> IResult<I,O>, O -> Result<P>) => I -> IResult<I, P>` maps a function returning a Result on the output of a parser
named!	Makes a function from a parser combination
none_of!	matches anything but the provided characters
none_of_bytes!
one_of!	matches one of the provided characters
one_of_bytes!
opt!	`opt!(I -> IResult<I,O>) => I -> IResult<I, Option<O>>` make the underlying parser optional
opt_res!	`opt_res!(I -> IResult<I,O>) => I -> IResult<I, Result<nom::Err,O>>` make the underlying parser optional
pair!	`pair!(I -> IResult<I,O>, I -> IResult<I,P>) => I -> IResult<I, (O,P)>` pair(X,Y), returns (x,y)
peek!	`peek!(I -> IResult<I,O>) => I -> IResult<I, O>` returns a result without consuming the input
preceded!	`preceded!(I -> IResult<I,T>, I -> IResult<I,O>) => I -> IResult<I, O>` preceded(opening, X) returns X
separated_list!	`separated_list!(I -> IResult<I,T>, I -> IResult<I,O>) => I -> IResult<I, Vec<O>>` separated_list(sep, X) returns Vec
separated_nonempty_list!	`separated_nonempty_list!(I -> IResult<I,T>, I -> IResult<I,O>) => I -> IResult<I, Vec<O>>` separated_nonempty_list(sep, X) returns Vec
separated_pair!	`separated_pair!(I -> IResult<I,O>, I -> IResult<I, T>, I -> IResult<I,P>) => I -> IResult<I, (O,P)>` separated_pair(X,sep,Y) returns (x,y)
switch!	`switch!(I -> IResult<I,P>, P => I -> IResult<I,O> \| ... \| P => I -> IResult<I,O> ) => I -> IResult<I, O>` choose the next parser depending on the result of the first one, if successful
tag!	`tag!(&[T]: nom::AsBytes) => &[T] -> IResult<&[T], &[T]>` declares a byte array as a suite to recognize
tag_bits!
tag_bytes!
take!	`take!(nb) => &[T] -> IResult<&[T], &[T]>` generates a parser consuming the specified number of bytes
take_bits!	`take_bits!(type, nb) => ( (&[T], usize), U, usize) -> IResult<(&[T], usize), U>` generates a parser consuming the specified number of bytes
take_str!	`take!(nb) => &[T] -> IResult<&[T], &str>` same as take! but returning a &str
take_till!	`take_till!(&[T] -> bool) => &[T] -> IResult<&[T], &[T]>` returns the longest list of bytes until the provided function succeeds
take_until!	`take_until!(tag) => &[T] -> IResult<&[T], &[T]>` consumes data until it finds the specified tag
take_until_and_consume!	`take_until_and_consume!(tag) => &[T] -> IResult<&[T], &[T]>` generates a parser consuming bytes until the specified byte sequence is found, and consumes it
take_until_and_consume_bytes!
take_until_bytes!
take_until_either!	`take_until_either!(tag) => &[T] -> IResult<&[T], &[T]>`
take_until_either_and_consume!	`take_until_either_and_consume!(tag) => &[T] -> IResult<&[T], &[T]>` consumes data until it finds any of the specified characters, and consume it
take_until_either_and_consume_bytes!
take_until_either_bytes!
take_while!	`take_while!(T -> bool) => &[T] -> IResult<&[T], &[T]>` returns the longest list of bytes until the provided function fails.
take_while1!	`take_while1!(&[T] -> bool) => &[T] -> IResult<&[T], &[T]>` returns the longest (non empty) list of bytes until the provided function fails.
tap!	`tap!(name: I -> IResult<I,O> => { block }) => I -> IResult<I, O>` allows access to the parser's result without affecting it
terminated!	`terminated!(I -> IResult<I,O>, I -> IResult<I,T>) => I -> IResult<I, O>` terminated(X, closing) returns X
try_parse!	A bit like `std::try!`, this macro will return the remaining input and parsed value if the child parser returned `Done`, and will do an early return for `Error` and `Incomplete` this can provide more flexibility than `chain!` if needed
u16!	if parameter is true, parse a big endian u16 integer, otherwise a little endian u16 integer
u32!	if parameter is true, parse a big endian u32 integer, otherwise a little endian u32 integer
u64!	if parameter is true, parse a big endian u64 integer, otherwise a little endian u64 integer

Structs

ChainConsumer	ChainConsumer takes a consumer C1 R -> S, and a consumer C2 S -> T, and makes a consumer R -> T by applying C2 on C1's result
FileProducer
MapConsumer	MapConsumer takes a function S -> T and applies it on a consumer producing values of type S
MemProducer	A MemProducer generates values from an in memory byte buffer
ProducerRepeat	ProducerRepeat takes a single value, and generates it at each step

Enums

ConsumerState	Stores a consumer's current computation state
Err	Contains the error that a parser can return
ErrorKind	indicates which parser returned an error
FileProducerState
IResult	Holds the result of parsing functions
Input
Move
Needed	Contains information on needed data if a parser returned `Incomplete`

Traits

AsBytes
Consumer	The Consumer trait wraps a computation and its state
GetInput
GetOutput
HexDisplay	useful functions to calculate the offset between slices and show a hexdump of a slice
InputLength
Producer	The producer wraps a data source, like file or network, and applies a consumer on it

Functions

add_error_pattern
alpha	Recognizes lowercase and uppercase alphabetic characters: a-zA-Z
alphanumeric	Recognizes numerical and alphabetic characters: 0-9a-zA-Z
anychar
be_f32	Recognizes big endian 4 bytes floating point number
be_f64	Recognizes big endian 8 bytes floating point number
be_i16	Recognizes big endian signed 2 bytes integer
be_i32	Recognizes big endian signed 4 bytes integer
be_i64	Recognizes big endian signed 8 bytes integer
be_i8	Recognizes a signed 1 byte integer (equivalent to take!(1)
be_u16	Recognizes big endian unsigned 2 bytes integer
be_u32	Recognizes big endian unsigned 4 bytes integer
be_u64	Recognizes big endian unsigned 8 bytes integer
be_u8	Recognizes an unsigned 1 byte integer (equivalent to take!(1)
begin
code_from_offset
compare_error_paths
crlf
digit	Recognizes numerical characters: 0-9
eof	Recognizes empty input buffers
eol
error_to_list
error_to_u32
generate_colors
hex_u32	Recognizes a hex-encoded integer
is_alphabetic
is_alphanumeric
is_digit
is_space
le_i16	Recognizes little endian signed 2 bytes integer
le_i32	Recognizes little endian signed 4 bytes integer
le_i64	Recognizes little endian signed 8 bytes integer
le_i8	Recognizes a signed 1 byte integer (equivalent to take!(1)
le_u16	Recognizes little endian unsigned 2 bytes integer
le_u32	Recognizes little endian unsigned 4 bytes integer
le_u64	Recognizes little endian unsigned 8 bytes integer
le_u8	Recognizes an unsigned 1 byte integer (equivalent to take!(1)
length_value
line_ending	Recognizes a line feed
multispace	Recognizes spaces, tabs, carriage returns and line feeds
newline
not_line_ending
prepare_errors
print
print_codes
print_error
print_offsets
reset_color
rest	Return the remaining input.
shift
sized_buffer
slice_to_offsets
space	Recognizes spaces and tabs
tab
tag_cl
write_color