A DOM fuzzer

转：https://github.com/google/domato

Written and maintained by Ivan Fratric, ifratric@google.com

Copyright 2017 Google Inc. All Rights Reserved.

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

Usage

To generate a single .html sample run:

python generator.py <output file>

To generate multiple samples with a single call run:

python generator.py --output_dir <output directory> --no_of_files <number of output files>

The generated samples will be placed in the specified directory and will be named as fuzz-<number>.html, e.g. fuzz-1.html, fuzz-2.html etc. Generating multiple samples is faster because the input grammar files need to be loaded and parsed only once.

Code organization

generator.py contains the main script. It uses grammar.py as a library and contains additional helper code for DOM fuzzing.

grammar.py contains the generation engine that is mostly application-agnostic//应用无关 and can thus be used in other (i.e. non-DOM) generation-based fuzzers. As it can be used as a library, its usage is described in a separate section below.

.txt files contain grammar definitions语法定义. There are 3 main files, html.txt, css.txt and js.txt which contain HTML, CSS and JavaScript grammars, respectively. These root grammar files may include content from other files.

Using the generation engine and writing grammars

To use the generation engine with a custom grammar, you can use the following python code:

from grammar import Grammar

my_grammar = Grammar()

my_grammar.parse_from_file('input_file.txt')

result_string = my_grammar.generate_symbol('symbol_name')

The following sections describe the syntax of the grammar files.

Basic syntax

Domato is based on an engine that, given a context-free grammar in a simple format specified below, generates samples from that grammar.

A grammar is described as a set of rules in the following basic format:

<symbol> = a mix of constants and <other_symbol>s

Each grammar rule contains a left side and the right side separated by the equal character. The left side contains a symbol, while the right side contains the details on how that symbol may be expanded. When expanding a symbol, all symbols on the right-hand side are expanded recursively while everything that is not a symbol is simply copied to the output. Note that a single rule can't span over multiple lines of the input file.

Consider the following simplified example of a part of the CSS grammar:

<cssrule> = <selector> { <declaration> }

<selector> = a

<selector> = b

<declaration> = width:100%

If we instruct the grammar engine to parse that grammar and generate 'cssrule', we may end up with either:

a { width:100% }

or

b { width:100% }

Note there are two rules for the 'selector' symbol. In such cases, when the generator is asked to generate a 'selector', it will select the rule to use at random. It is also possible to specify the probability of the rule using the 'p' attribute, for example:

<selector p=0.9> = a

<selector p=0.1> = b

In this case, the string 'a' would be output more often than 'b'.

There are other attributes that can be applied to symbols in addition to the probability. Those are listed in a separate section.

Consider another example for generating html samples:

<html> = <lt>html<gt><head><body><lt>/html<gt>

<head> = <lt>head<gt>...<lt>/head<gt>

<body> = <lt>body<gt>...<lt>/body<gt>

Note that since the '<' and '>' have a special meaning in the grammar syntax, so here we are using <lt> and <gt> instead. These symbols are built in and don't need to be defined by the user. A list of all built-in symbols is provided in a separate section.

Generating programming language code

To generate programming language code, a similar syntax can be used, but there are a couple of differences. Each line of the programming language grammar is going to correspond to the line of the output. Because of that, the grammar syntax is going to be more free-form to allow expressing constructs in various programming languages. Secondly, when a line is generated, in addition to outputting the line, one or more variables may be created and those variables may be reused when generating other lines. Again, let's take a look of the simplified example:

!varformat fuzzvar%05d//定义变量名称格式

!lineguard try { <line> } catch(e) {}//异常处理

!begin lines

<new element> = document.getElementById("<string min=97 max=122>");

<element>.doSomething();

!end lines

If we instruct the engine to generate 5 lines, we may end up with something like:

try { var00001 = document.getElementById("hw"); } catch(e) {}

try { var00001.doSomething(); } catch(e) {}

try { var00002 = document.getElementById("feezcqbndf"); } catch(e) {}

try { var00002.doSomething(); } catch(e) {}

try { var00001.doSomething(); } catch(e) {}

Note that

  • programming language lines are enclosed in '!begin lines' and '!end lines' statement. This gives the grammar parser the necessary information that the lines in-between are programming language lines and are thus parsed differently.
  • We used <new element> instead of <element>. This instructs the generator to create a new variable of type 'element' instead of generating the 'element' symbol.
  • <string> is one of the built-in symbols so no need to define it.
  • [optional] You can use !varformat statement to define the format of variables you want to use.
  • [optional] You can use !lineguard statement to define additional code that gets inserted around every line in order to catch exceptions or perform other tasks. This is so you wouldn't need to write it for every line separately.
  • In addition to '!begin lines' and '!end lines' you can also use '!begin helperlines' and '!end helperlines' to define lines of code that will only ever be used if required when generating other lines (for example, helper lines might generate variables needed by the 'main' code, but you don't ever want those helper lines to end up in the output when they are not needed).
Comments

Everything after the first '#' character on the line is considered a comment, so for example:

#This is a comment
Preventing infinite recursions

The grammar syntax has a way of telling the fuzzer which rules are nonrecursive and can be safe to use even if the maximum level of recursion has been reached. This is done with the ‘nonrecursive’ attributes. An example is given below.

!max_recursion 10

<test root=true> = <foobar>

<foobar> = foo<foobar>

<foobar nonrecursive> = bar

Firstly, an optional ‘!max_recursion’ statement defines the maximum recursion depth level (50 by default). Notice that the second production rule for ‘foobar’ is marked as non-recursive. If ever the maximum recursion level is reached the generator will force using the non-recursive rule for ‘foobar’ symbol, thus preventing infinite recursion.

Including and importing other grammar files

In Domato, including and importing grammars are two different context.

Including is simpler. You can use:

!include other.txt

to include rules from other.txt into the currently parsed grammar.

Importing works a bit differently:

!import other.txt

tells the parser to create a new Grammar() object that can then be referenced from the current grammar by using the special <import> symbol, for example like this:

<cssrule> = <import from=css.txt symbol=rule>

You can think about importing and including in terms of namespaces: !include will put the included grammar into the single namespace, while !import will create a new namespace which can then be accessed using the <import> symbol and the namespace specified via the 'from' attribute.

Including Python code

Sometimes you might want to call custom Python code in your grammar. For example, let’s say you want to use the engine to generate a http response and you want the body length to match the 'Size' header. Since this is something not possible with normal grammar rules, you can include custom Python code to accomplish it like this:

!begin function savesize

  context['size'] = ret_val

!end function

!begin function createbody

  n = int(context['size'])

  ret_val = 'a' * n

!end function

<foo root> = <header><cr><lf><body>

<header> = Size: <int min=1 max=20 beforeoutput=savesize>

<body> = <call function=createbody>

The python functions are defined between ‘!begin function <function_name>’ and ‘!end function’ commands. The functions can be called in two ways: using ‘beforeoutput’ attribute and using symbol.

By specifying the ‘beforeoutput’ attribute in some symbol, the corresponding function will be called when this symbol is expanded, just before the result of the expansion is output to the sample. The expansion result will be passed to the function in the ret_val variable. The function is then free to modify ret_val, store it for later use or perform any other operations.

When using a special <call> symbol, the function (specified in a ‘function’ attribute) will be called when the symbol is encountered during language generation. Any value stored by the function in ret_val will be considered the result of the expansion (ret_val gets included in the sample).

Your python code has access to the following variables:

  • context - a dictionary that is passed through the whole sample generation. You can use it to store values (such as storing the size in an example above) and retrieve them in the rules that fire subsequently.
  • attributes - a dictionary corresponding to the symbol currently being processed. You can use it to pass parameters to your functions. For example if you used something like to call your function attributes[‘foo’] will be set to ‘bar’.
  • ret_val - The value that will be output as a result of the function call. It is initialized to an empty value when using symbol to call a function, otherwise it will be initialized to the value generated by the symbol.
Built-in symbols

The following symbols have a special meaning and should not be redefined by users:

  • <lt> - ‘<’ character
  • <gt> - ‘>’ character
  • <hash> - ‘#’ character
  • <cr> - CR character
  • <lf> - LF character
  • <space> - space character
  • <tab> - tab character
  • <ex> - ‘!’ character
  • <char> - can be used to generate an arbitrary ascii character using ‘code’ attribute. For example <char code=97>corresponds to ‘a’. Generates random character if not specified. Supports ‘min’ and ‘max’ attribute.
  • <hex> - generates a random hex digit.
  • <int>, <int 8>, <uint8>, <int16>, <uint16>, <int32>, <uint32>, <int64>, <uint64> - can be used to generate random integers. Supports ‘min’ and ‘max’ attribute that can be used to limit the range of integers that will be generated. Supports the ‘b’ and ‘be’ attribute which makes the output binary in little/big endian format instead of text output.
  • <float>, <double> - generates a random floating-point number. Supports ‘min’ and ‘max’ attribute (0 and 1 if not specified). Supports ‘b’ attribute which makes the output binary.
  • <string> - generates a random string. Supports ‘min’ and ‘max’ attributes which control the minimum and maximum charcode generated as well as ‘minlength’ and ‘maxlength’ attributes that control the length of the string.
  • <htmlsafestring> - same as <string> except HTML metacharacters will be escaped, making it safe to embed the string as part of HTML text or attribute values.
  • <lines> - outputs the given number (via ‘count’ attribute) lines of code. See the section on generating programming language code for example.
  • <import> - imports a symbol from another grammar, see the section on including external grammars for details.
  • <call> - calls a user-defined function corresponding to the function attribute. See the section on including Python code in the grammar for more info.
Symbol attributes

The following attributes are supported:

  • root - marks a symbol as the root symbol of the grammar. The only supported value is ‘true’. When GenerateSymbol() is called, if no argument is specified, the root symbol will be generated.
  • nonrecursive - gives the generator a hint that this rule doesn’t contain recursion loops and is used to prevent infinite recursions. The only supported value is ‘true’.
  • new - used when generating programming languages to denote that a new variable is created here rather than expanding the symbol as usual. The only supported value is ‘true’.
  • from, symbol - used when importing symbols from other grammars, see ‘Including external grammars’ section.
  • count - used in lines symbol to specify the number of lines to be created.
  • id - used to mark that several symbols should share the same value. For example in the rule ‘doSomething(<int id=1>, <int id=1>)’ both ints would end up having the same value. Only the first instance is actually expanded, the second is just copied from the first.
  • min, max - used in generation of numeric types to specify the minimum and maximum value. Also used to limit the set of characters generated in strings.
  • b, be - used in numeric types to specify binary little-endian (‘b’) or big-endian (‘be’) output.
  • code - used in char symbol to specify the exact character to output by its code.
  • minlength, maxlength - used when generating strings to specify the minimum and maximum length.
  • up - used in hex symbol to specify uppercase output (lowercase is the default).
  • function - used in the <call> symbol, see ‘Including Python code’ section for more info.
  • beforeoutput - used to call user-specified functions, see ‘Including Python’.

Bug Showcase

Some of the bugs that have been found with Domato:

  • Apple Safari: CVE-2017-2369, CVE-2017-2373, CVE-2017-2362, CVE-2017-2454, CVE-2017-2455, CVE-2017-2459, CVE-2017-2460, CVE-2017-2466, CVE-2017-2471, CVE-2017-2476, CVE-2017-7039, CVE-2017-7040, CVE-2017-7041, CVE-2017-7042, CVE-2017-7043, CVE-2017-7046, CVE-2017-7048, CVE-2017-7049
  • Google Chrome: Issues 666246 and 671328
  • Microsoft Internet Explorer 11: CVE-2017-0037, CVE-2017-0059, CVE-2017-0202, CVE-2017-8594
  • Microsoft Edge: CVE-2017-0037, CVE-2017-8496, CVE-2017-8652, CVE-2017-8644
  • Mozilla Firefox: CVE-2017-5404, CVE-2017-5447, CVE-2017-5465

Disclaimer

This is not an official Google product.

Domato学习的更多相关文章

  1. 从直播编程到直播教育：LiveEdu.tv开启多元化的在线学习直播时代

    2015年9月,一个叫Livecoding.tv的网站在互联网上引起了编程界的注意.缘于Pingwest品玩的一位编辑在上网时无意中发现了这个网站,并写了一篇文章<一个比直播睡觉更奇怪的网站:直 ...

  2. Angular2学习笔记（1）

    Angular2学习笔记(1) 1. 写在前面 之前基于Electron写过一个Markdown编辑器.就其功能而言,主要功能已经实现,一些小的不影响使用的功能由于时间关系还没有完成:但就代码而言,之 ...

  3. ABP入门系列（1）——学习Abp框架之实操演练

    作为.Net工地搬砖长工一名,一直致力于挖坑(Bug)填坑(Debug),但技术却不见长进.也曾热情于新技术的学习,憧憬过成为技术大拿.从前端到后端,从bootstrap到javascript,从py ...

  4. 消息队列——RabbitMQ学习笔记

    消息队列--RabbitMQ学习笔记 1. 写在前面 昨天简单学习了一个消息队列项目--RabbitMQ,今天趁热打铁,将学到的东西记录下来. 学习的资料主要是官网给出的6个基本的消息发送/接收模型, ...

  5. js学习笔记：webpack基础入门（一）

    之前听说过webpack,今天想正式的接触一下,先跟着webpack的官方用户指南走: 在这里有: 如何安装webpack 如何使用webpack 如何使用loader 如何使用webpack的开发者 ...

  6. Unity3d学习 制作地形

    这周学习了如何在unity中制作地形,就是在一个Terrain的对象上盖几座小山,在山底种几棵树,那就讲一下如何完成上述内容. 1.在新键得项目的游戏的Hierarchy目录中新键一个Terrain对 ...

  7. 《Django By Example》第四章 中文 翻译 （个人学习，渣翻）

    书籍出处:https://www.packtpub.com/web-development/django-example 原作者:Antonio Melé (译者注:祝大家新年快乐,这次带来<D ...

  8. 菜鸟Python学习笔记第一天：关于一些函数库的使用

    2017年1月3日 星期二 大一学习一门新的计算机语言真的很难,有时候连函数拼写出错查错都能查半天,没办法,谁让我英语太渣. 关于计算机语言的学习我想还是从C语言学习开始为好,Python有很多语言的 ...

  9. 多线程爬坑之路-学习多线程需要来了解哪些东西?(concurrent并发包的数据结构和线程池,Locks锁，Atomic原子类)

    前言:刚学习了一段机器学习,最近需要重构一个java项目,又赶过来看java.大多是线程代码,没办法,那时候总觉得多线程是个很难的部分很少用到,所以一直没下决定去啃,那些年留下的坑,总是得自己跳进去填 ...

随机推荐

  1. JAVA Remote Object

    RMI(Remote Method Invocation)远程方法调用是一种计算机之间利用远程对象相互调用实现双方通讯的一种通讯机制.使用这种机制,某一台计算机上的对象可以调用另一台计算机上的对象来获 ...

  2. 几何+思维 Samara University ACM ICPC 2016-2017 Quarterfinal Qualification Contest K. Revenge of the Dragon

    题目链接:http://codeforces.com/gym/101149/problem/K 题目大意: 给你两个点a,b.一个人在a点,一个人在b点,b点的人要追杀a的点,他的跑步速度是a的两倍. ...

  3. Zepto学习笔记

    Zepto和jQuery的很多API都很一致,思路也很相似,我不会全都整理出来,只是把一些平时用到了的或者不同的地方需要注意一下的地方总结出来.另外,Zepto现在还不是很成熟,无论是对大小写的敏感还 ...

  4. Tool1—安装配置Windows Live Writer

    详细步骤请看:http://home.cnblogs.com/group/topic/8550.html . Windows Live Writer手工配置步骤(在博客园配置时输入用户名与密码会自动完 ...

  5. 【BZOJ】1711: [Usaco2007 Open]Dining吃饭

    [算法]最大流 [题解] S连向食物连向牛连向牛‘连向饮料连向T. 经典的一个元素依赖于两个元素的建图方式. #include<cstdio> #include<algorithm& ...

  6. 数组B - 我想我需要一艘船屋

    [题目大意]弗雷德先生正在考虑在路易斯安娜州买一块地造房子,在土地调查中,他了解到由于密西西比河的侵蚀,路易斯安那州正以每年50平方英里的速度变小.弗雷德先生想知道他买的那块地是否会被侵蚀掉,经过进一 ...

  7. yii2 自动登录解读

    今日遇到一个需要将当前用户,全部登出系统(YII2框架制作)重新登录的需求 仔细回忆一遍,Yii2的登录流程,竟然有些不太明白,于是下午空闲时 重新看了下Yii2的用户登录源码 文件位于YII2项目下 ...

  8. cookie、localstroage与sessionstroage的一些优缺点

    1.    Cookie 在前端开发中,尽量少用cooie,原因: (1)   cookie限制大小,约4k左右,不适合存储业务数据,尤其是数据量较大的值: (2)   cookie会每次随http请 ...

  9. Dull Chocolates Gym - 101991D 离散化 前缀和

    题目链接:https://vjudge.net/problem/Gym-101991D 具体思路:首先看数据范围,暴力肯定不可以,可以下离散化,然后先求出离散化后每一个点到(1,1)的符合题目的要求的 ...

  10. nginx 配置代理某个路径

    location /test{ proxy_pass http://localhost:8765/test; proxy_set_header Host $http_host; } 其中红色的那句可以 ...