logo
down
shadow

Simple expression parser example using Boost::Spirit?


Simple expression parser example using Boost::Spirit?

By : fred
Date : October 21 2020, 11:12 AM
I wish did fix the issue. Is anyone aware of an online resource where I can find out how to write a simple expression parser using Boost::Spirit?. , Here's some old Spirit prototype code I had laying around:
code :


Share : facebook icon twitter icon
What is the problem with this simple boost::spirit::qi parser?

What is the problem with this simple boost::spirit::qi parser?


By : user2101600
Date : March 29 2020, 07:55 AM
I think the issue was by ths following , I've got this simple parser intended to parse VB style double quoted strings. Thus, the parser should turn something like , You can do it without any semantic actions:
code :
class ConfigurationParser 
  : public qi::grammar<std::wstring::iterator, std::wstring()> 
{ 
    qi::rule<std::wstring::iterator, std::wstring()> quotedString; 
    qi::rule<std::wstring::iterator, wchar_t()> doubleQuote; 

public: 
    ConfigurationParser() 
      : ConfigurationParser::base_type(quotedString, "vFind Command Line") 
    { 
        doubleQuote = wide::char_(L'"') >> omit[wide::char_(L'"')]; 
        quotedString = L'"' >> +(doubleQuote | (wide::char_ - L'"')) >> L'"'; 
    } 
}; 
Boost::Spirit Expression Parser

Boost::Spirit Expression Parser


By : user3877225
Date : March 29 2020, 07:55 AM
This might help you It isn't entirely clear to me what you are trying to achieve. Most importantly, are you not worried about operator associativity? I'll just show simple answers based on using right-recursion - this leads to left-associative operators being parsed.
The straight answer to your visible question would be to juggle a fusion::vector2 - which isn't really any fun, especially in Phoenix lambda semantic actions. (I'll show below, what that looks like).
code :
std::string input("1/2+3-4*5");
<expr>
  ....
  <success></success>
  <attributes>[[1, [2, [3, [4, 5]]]]]</attributes>
</expr>
static ast::expression make_binop(char discriminant, 
     const ast::expression& left, const ast::expression& right)
{
    switch(discriminant)
    {
        case '+': return ast::binary_op<ast::add>(left, right);
        case '-': return ast::binary_op<ast::sub>(left, right);
        case '/': return ast::binary_op<ast::div>(left, right);
        case '*': return ast::binary_op<ast::mul>(left, right);
    }
    throw std::runtime_error("unreachable in make_binop");
}

// rules:
number %= lexeme[double_];
varname %= lexeme[alpha >> *(alnum | '_')];

simple = varname | number;
binop = (simple >> char_("-+*/") >> expr) 
    [ _val = phx::bind(make_binop, qi::_2, qi::_1, qi::_3) ]; 

expr = binop | simple;
number %= lexeme[double_];
varname %= lexeme[alpha >> *(alnum | '_')];

simple = varname | number;
expr = simple [ _val = _1 ] 
    > *(char_("-+*/") > expr) 
            [ _val = phx::bind(make_binop, qi::_1, _val, qi::_2) ]
    > eoi;
static ast::expression make_binop(
        const ast::expression& left, 
        const boost::fusion::vector2<char, ast::expression>& op_right)
{
    switch(boost::fusion::get<0>(op_right))
    {
        case '+': return ast::binary_op<ast::add>(left, boost::fusion::get<1>(op_right));
        case '-': return ast::binary_op<ast::sub>(left, boost::fusion::get<1>(op_right));
        case '/': return ast::binary_op<ast::div>(left, boost::fusion::get<1>(op_right));
        case '*': return ast::binary_op<ast::mul>(left, boost::fusion::get<1>(op_right));
    }
    throw std::runtime_error("unreachable in make_op");
}

// rules:
expression::base_type(expr) {
number %= lexeme[double_];
varname %= lexeme[alpha >> *(alnum | '_')];

simple = varname | number;
binop %= (simple >> (char_("-+*/") > expr)) 
    [ _val = phx::bind(make_binop, qi::_1, qi::_2) ]; // note _2!!!

expr %= binop | simple;
Compiling a simple parser with Boost.Spirit

Compiling a simple parser with Boost.Spirit


By : Andrey Tsiganok
Date : March 29 2020, 07:55 AM
will help you Mmm. I feel that we have discussed a few more details in chat than have been reflected in the question as it is.
Let me entertain you with my 'toy' implementation, complete with test cases, of a grammar that will recognize <> like this, including nested expansion of the same.
code :
//#define BOOST_SPIRIT_DEBUG
#include <boost/spirit/include/qi.hpp>
#include <boost/spirit/include/phoenix.hpp>

namespace qi = boost::spirit::qi;
namespace phx= boost::phoenix;
namespace fsn= boost::fusion;

namespace
{
    #define SUPPORT_ESCAPES

    static bool process(std::string& macro)
    {
        if (macro == "error") {
            return false; // fail the parse
        }

        if (macro == "hello") {
            macro = "bye";
        } else if (macro == "bye") {
            macro = "We meet again";
        } else if (macro == "sideeffect") {
            std::cerr << "this is a side effect while parsing\n";
            macro = "(done)";
        } else if (std::string::npos != macro.find('~')) {  
            std::reverse(macro.begin(), macro.end());
            macro.erase(std::remove(macro.begin(), macro.end(), '~'));
        } else {
            macro = std::string("<<") + macro + ">>"; // this makes the unsupported macros appear unchanged
        }

        return true;
    }

    template<typename Iterator, typename OutIt>
        struct skel_grammar : public qi::grammar<Iterator>
    {
        struct fastfwd {
            template<typename,typename> struct result { typedef bool type; };

            template<typename R, typename O> 
                bool operator()(const R&r,O& o) const
            {
#ifndef SUPPORT_ESCAPES
                o = std::copy(r.begin(),r.end(),o);
#else
                auto f = std::begin(r), l = std::end(r);
                while(f!=l)
                {
                    if (('\\'==*f) && (l == ++f))
                        break;
                    *o++ = *f++;
                }
#endif
                return true; // false to fail the parse
            }
        } copy;

        skel_grammar(OutIt& out) : skel_grammar::base_type(start)
        {
            using namespace qi;

#ifdef SUPPORT_ESCAPES
            rawch = ('\\' >> char_) | char_;
#else
#           define rawch qi::char_
#endif

            macro = ("<<" >> (
                           (*(rawch - ">>" - "<<") [ _val += _1 ]) 
                         % macro                   [ _val += _1 ] // allow nests
                      ) >> 
                      ">>")  
                [ _pass = phx::bind(process, _val) ];

            start = 
                raw [ +(rawch - "<<") ] [ _pass = phx::bind(copy, _1, phx::ref(out)) ] 
              % macro                   [ _pass = phx::bind(copy, _1, phx::ref(out)) ]
              ;

            BOOST_SPIRIT_DEBUG_NODE(start);
            BOOST_SPIRIT_DEBUG_NODE(macro);


#           undef rawch
        }

        private:
#ifdef SUPPORT_ESCAPES
        qi::rule<Iterator, char()> rawch;
#endif
        qi::rule<Iterator, std::string()> macro;
        qi::rule<Iterator> start;
    };
}

int main(int argc, char* argv[])
{
    std::string input = 
        "Greeting is <<hello>> world!\n"
        "Side effects are <<sideeffect>> and <<other>> vars are untouched\n"
        "Empty <<>> macros are ok, as are stray '>>' pairs.\n"
        "<<nested <<macros>> (<<hello>>?) work>>\n"
        "The order of expansion (evaluation) is _eager_: '<<<<hello>>>>' will expand to the same as '<<bye>>'\n"
        "Lastly you can do algorithmic stuff too: <<!esrever ~ni <<hello>>>>\n"
#ifdef SUPPORT_ESCAPES // bonus: escapes
        "You can escape \\<<hello>> (not expanded to '<<hello>>')\n"
        "Demonstrate how it <<avoids <\\<nesting\\>> macros>>.\n"
#endif
        ;

    std::ostringstream oss;
    std::ostream_iterator<char> out(oss);

    skel_grammar<std::string::iterator, std::ostream_iterator<char> > grammar(out);

    std::string::iterator f(input.begin()), l(input.end());
    bool r = qi::parse(f, l, grammar);

    std::cout << "parse result: " << (r?"success":"failure") << "\n";
    if (f!=l)
        std::cout << "unparsed remaining: '" << std::string(f,l) << "'\n";

    std::cout << "Streamed output:\n\n" << oss.str() << '\n';

    return 0;
}
this is a side effect while parsing
parse result: success
Streamed output:

Greeting is bye world!
Side effects are (done) and <<other>> vars are untouched
Empty <<>> macros are ok, as are stray '>>' pairs.
<<nested <<macros>> (bye?) work>>
The order of expansion (evaluation) is _eager_: 'We meet again' will expand to the same as 'We meet again'
Lastly you can do algorithmic stuff too: eyb in reverse!
You can escape <<hello>> (not expanded to 'bye')
Demonstrate how it <<avoids <<nesting>> macros>>.
Boost::Spirit : Optimizing an expression parser

Boost::Spirit : Optimizing an expression parser


By : Xa4yPiva
Date : March 29 2020, 07:55 AM
This might help you Your problem is that (1) is the worst possible case for backtracking with that grammar. Let's study a simplified example:
code :
or_ = (and_ >> '|' >> or_) | and_;
and_ = (not_ >> '&' >> and_) | not_;
not_ = ('!' >> simple_) | simple_;
simple_ = ('(' >> or_ >> ')') | var_;
or_ = and_[_val=_1] >> -( '|' >> or_ )[_val=construct<binop<op_or> >(_val,_1)]; 
or_ = and_[_val=_1] >> *( '|' >> and_)[_val=construct<binop<op_or> >(_val,_1)]; //note the `and_` instead of `or_` after '|'
or_ = fold<binop<op_or> >(xor_.alias())['|' >> xor_]; //sadly the `.alias()` is required
namespace custom
{
    namespace tag
    {
        struct fold { BOOST_SPIRIT_IS_TAG() };
    }

    template <typename Exposed, typename Expr>
    boost::spirit::stateful_tag_type<Expr, tag::fold, Exposed>
    fold(Expr const& expr)
    {
        return boost::spirit::stateful_tag_type<Expr, tag::fold, Exposed>(expr);
    }

}

namespace boost { namespace spirit 
{
    template <typename Expr, typename Exposed>
    struct use_directive<qi::domain
          , tag::stateful_tag<Expr, custom::tag::fold, Exposed> >
      : mpl::true_ {};
}}

namespace custom
{
    template <typename Exposed, typename InitialParser, typename RepeatingParser>
    struct fold_directive
    {
        fold_directive(InitialParser const& initial, RepeatingParser const& repeating):initial(initial),repeating(repeating){}

        template <typename Context, typename Iterator>
        struct attribute
        {
            typedef typename boost::spirit::traits::attribute_of<InitialParser,Context,Iterator>::type type;//This works in this case but is not generic
        };

        template <typename Iterator, typename Context
          , typename Skipper, typename Attribute>
        bool parse(Iterator& first, Iterator const& last
          , Context& context, Skipper const& skipper, Attribute& attr_) const
        {
            Iterator start = first;

            typename boost::spirit::traits::attribute_of<InitialParser,Context,Iterator>::type initial_attr;


            if (!initial.parse(first, last, context, skipper, initial_attr))
            {
                first=start;
                return false;
            }

            typename boost::spirit::traits::attribute_of<RepeatingParser,Context,Iterator>::type repeating_attr;

            if(!repeating.parse(first, last, context, skipper, repeating_attr))
            {
                boost::spirit::traits::assign_to(initial_attr, attr_);
                return true;
            }
            Exposed current_attr(initial_attr,repeating_attr);

            while(repeating.parse(first, last, context, skipper, repeating_attr))
            {
                boost::spirit::traits::assign_to(Exposed(current_attr,repeating_attr),current_attr);
            }
            boost::spirit::traits::assign_to(current_attr,attr_);
            return true;
        }

        template <typename Context>
        boost::spirit::info what(Context& context) const
        {
            return boost::spirit::info("fold");
        }

        InitialParser initial;
        RepeatingParser repeating;
    };
}

namespace boost { namespace spirit { namespace qi
{
    template <typename Expr, typename Exposed, typename Subject, typename Modifiers>
    struct make_directive<
        tag::stateful_tag<Expr, custom::tag::fold, Exposed>, Subject, Modifiers>
    {
        typedef custom::fold_directive<Exposed, Expr, Subject> result_type;

        template <typename Terminal>
        result_type operator()(Terminal const& term, Subject const& subject, Modifiers const&) const
        {
            typedef tag::stateful_tag<
                Expr, custom::tag::fold, Exposed> tag_type;
            using spirit::detail::get_stateful_data;

            return result_type(get_stateful_data<tag_type>::call(term),subject);
        }
    };
}}}
//#define BOOST_SPIRIT_DEBUG
#include <iostream>
#include <boost/spirit/include/qi.hpp>
#include <boost/spirit/include/phoenix.hpp>
#include "custom_fold_directive.hpp"

namespace qi = boost::spirit::qi;


// DEFINING TYPES
struct op_not {};
struct op_or {};
struct op_and {};
struct op_xor {};
struct op_equal {};
struct op_unequal {};
struct op_sum {};
struct op_difference {};
struct op_factor {};
struct op_division {};

namespace Expression{

typedef std::string var;
template <typename tag> struct binop;
template <typename tag> struct unop;

typedef boost::variant<var,
    boost::recursive_wrapper<unop <op_not> >,
    boost::recursive_wrapper<binop<op_equal> >,
    boost::recursive_wrapper<binop<op_unequal> >,
    boost::recursive_wrapper<binop<op_and> >,
    boost::recursive_wrapper<binop<op_xor> >,
    boost::recursive_wrapper<binop<op_or> >,
    boost::recursive_wrapper<binop<op_difference> >,
    boost::recursive_wrapper<binop<op_sum> >,
    boost::recursive_wrapper<binop<op_factor> >,
    boost::recursive_wrapper<binop<op_division> >
> expressionContainer;


template <typename tag> struct binop
{
    explicit binop(const expressionContainer& l
        , const expressionContainer& r)
        : oper1(l), oper2(r) { }
    expressionContainer oper1, oper2;

    friend std::ostream& operator<<(std::ostream& os, const binop& val)
    {
        os << "(" << typeid(tag).name() << " " << val.oper1 << ", "<< val.oper2 << ")";
        return os;
    }
};

template <typename tag> struct unop
{
    explicit unop(const expressionContainer& o) : oper1(o) { }
    expressionContainer oper1;

    friend std::ostream& operator<<(std::ostream& os, const unop& val)
    {
        os << "(" << typeid(tag).name() << " " << val.oper1 << ")";
        return os;
    }
};

}

    // EXPRESSION PARSER
template <typename It, typename Skipper = boost::spirit::standard_wide::space_type>
struct parserExpression : qi::grammar<It, Expression::expressionContainer(), Skipper>
{
    parserExpression() : parserExpression::base_type(expr_)
    {
        using namespace qi;
        using namespace Expression;
        using custom::fold;

        expr_ = or_.alias();

        // Logical Operators
        or_ = fold<binop<op_or> >(xor_.alias())[orOperator_ >> xor_];
        xor_ = fold<binop<op_xor> >(and_.alias())[xorOperator_ >> and_];
        and_ = fold<binop<op_and> >(equal_.alias())[andOperator_ >> equal_];
        equal_ = fold<binop<op_equal> >(unequal_.alias())[equalOperator_ >> unequal_]; 
        unequal_ = fold<binop<op_unequal> >(sum_.alias())[unequalOperator_ >> sum_];

        // Numerical Operators
        sum_ = fold<binop<op_sum> >(difference_.alias())[sumOperator_ >> difference_];
        difference_ = fold<binop<op_difference> >(factor_.alias())[differenceOperator_ >> factor_];
        factor_ = fold<binop<op_factor> >(division_.alias())[factorOperator_ >> division_]; 
        division_ = fold<binop<op_division> >(not_.alias())[divisionOperator_ >> not_];

        // UNARY OPERATIONS
        not_ = (notOperator_ > simple) [_val = boost::phoenix::construct<Expression::unop <op_not>>(_1)] | simple[_val = _1];

        simple = (('(' > expr_ > ')') | var_);
        var_ = qi::lexeme[+alnum];

        notOperator_        = qi::char_('!');
        andOperator_        = qi::string("&&");
        orOperator_         = qi::string("||");
        xorOperator_        = qi::char_("^");
        equalOperator_      = qi::string("==");
        unequalOperator_    = qi::string("!=");
        sumOperator_        = qi::char_("+");
        differenceOperator_ = qi::char_("-");
        factorOperator_     = qi::char_("*");
        divisionOperator_   = qi::char_("/");

        BOOST_SPIRIT_DEBUG_NODES((expr_)(or_)(xor_)(and_)(equal_)(unequal_)(sum_)(difference_)(factor_)(division_)(simple)(notOperator_)
                                 (andOperator_)(orOperator_)(xorOperator_)(equalOperator_)(unequalOperator_)(sumOperator_)(differenceOperator_)(factorOperator_)(divisionOperator_));

    }

private:
    qi::rule<It, Expression::var(), Skipper> var_;
    qi::rule<It, Expression::expressionContainer(), Skipper> not_
        , and_
        , xor_
        , or_
        , equal_
        , unequal_
        , sum_
        , difference_
        , factor_
        , division_
        , simple
        , expr_;

    qi::rule<It, Skipper> notOperator_
        , andOperator_
        , orOperator_
        , xorOperator_
        , equalOperator_
        , unequalOperator_
        , sumOperator_
        , differenceOperator_
        , factorOperator_
        , divisionOperator_;
};

void parse(const std::string& str)
{
    std::string::const_iterator iter = str.begin(), end = str.end();

    parserExpression<std::string::const_iterator,qi::space_type> parser;
    Expression::expressionContainer expr;

    bool result = qi::phrase_parse(iter,end,parser,qi::space, expr);

    if(result && iter==end)
    {
        std::cout << "Success." << std::endl;
        std::cout << str << " => " << expr << std::endl;
    }
    else
    {
        std::cout << "Failure." << std::endl;
    }
}

int main()
{
    parse("(1)");
    parse("3-2-1");
    parse("a+b*c");
    parse("a*b+c");
    parse("(a+b)*c");
    parse("a*b+c*(d+e)&&true||false");
}
Simple string parser in boost::spirit::x3 not working

Simple string parser in boost::spirit::x3 not working


By : Christian
Date : March 29 2020, 07:55 AM
With these it helps I think that's an issue that has been fixed:
boost 1.66 fails https://wandbox.org/permlink/H1InadjEYI8KTHto boost 1.67 succeeds https://wandbox.org/permlink/aZnnpzizXFOe7emp
Related Posts Related Posts :
  • `LoadLibraryExW` triggers exception `0xC0000023` from `NtMapViewOfSection`
  • static_cast to a struct type to access all of its member variable
  • Forward declaration for boost::intrusive_ptr class to reduce compile time
  • How to use priority_queue with a non-static compare method of class instance?
  • Template parameters inside and outside class/struct
  • Determining prime number
  • How to resolve ambiguity between constructors taking std::string and std::vector
  • My program crashes when I try to change private values from an object
  • Unordered_map with custom class as key
  • Strict aliasing rules broken with templates and inheritance
  • C++ Derived Class Override Return Type
  • singly linked list c++ constructor, destructor and printing out
  • How to clone class with vector of unique_ptr to base class
  • error: no match for operator
  • std::vector doesnt accept my struct as template
  • selection of people's contours
  • how to fix the (Error using mexOpenCV) on matlab?
  • Is or was there a proposal for c++ to use the context for short enum values?
  • Fair assumptions about std::hash implementations
  • undefined reference to libusb using cyusb
  • Function returns null pointer instead of address
  • C++17 copy elision and object destruction
  • Input multiple strings via operator>> c++
  • Avoiding overflow boost container
  • How to Write a Lambda Wrapping a Function with Optional Return Value
  • Partial specialization with more template parameters
  • How to convert fixed size array to pointer on pointer array
  • Memory leak in const member constructor with tag dispatching
  • C++ function with a generic and optional class
  • Custom QGraphicsItem That Contains Child QGraphicsItems
  • Are There Restrictions on What can be Passed to auto Template Parameters?
  • Rotating line inside rectangle bounds
  • Why do I need dynamic memory allocation if I can just create an array?
  • How can I convert a text file into a form that MPI_Bcast can send?
  • How to get array of all `this` of an instance
  • Using pointers as parameters
  • Automatic type deduction with const_cast is not working
  • Why does std::is_rvalue_reference not do what it is advertised to do?
  • Function Template Specialization with Forward Declared Type
  • template deduction failed in vector
  • Is there a signed `sizeof` alternative in Qt
  • clarification on overloading the -> operator
  • What is __m128d?
  • QtConcurrent: why releaseThread and reserveThread cause deadlock?
  • Function receiving different value than passed
  • Can C++ close a '''fstream''' variable after '''.close()'''?
  • Is it necessary to overload operator in this specific case
  • Comparing an element of a string array with a string
  • how to dereference a pointer of a map of pointers to objects in c++
  • How recursive function control flow change for static variable?
  • SDL 2 blitting BMP File
  • Why does an extremely large value cause this code to repeat infinitely?
  • Function returning different value than what is in function body before return statement
  • Struggling with including Headers in another header file (C++)
  • WebViewProcessControl initialization crash
  • C++ exception 0xC00000FD: Stack overflow (parameters: 0x00000001, 0x00C02F30)
  • What's difference between forward and move in the constructor of class?
  • C++: Is it possible to condense `bool` objects within the same byte?
  • I cant understand this access modifier
  • Boost Spirit X3: Collapsing one-element lists
  • shadow
    Privacy Policy - Terms - Contact Us © bighow.org