cEP-0018: Integration of ANTLR in coala

Propose integration of ANTLR in coala.

Closes #118

cEP-0018.md

@@ -0,0 +1,166 @@
+# Integration of ANTLR into coala core
+
+| Metadata |                                               |
+| -------- | --------------------------------------------- |
+| cEP      | 18                                            |
+| Version  | 1.0                                           |
+| Title    | Integration of ANTLR into coala core          |
+| Authors  | Viresh Gupta <mailto:[email protected]> |
+| Status   | Proposed                                      |
+| Type     | Feature                                       |
+
+## Abstract
+
+This document describes how an API based on ANTLR will be constructed and
+maintained.
+
+## Introduction
+
+ANTLR provides parsers for various language grammars and thus we can provide
+an interface from within coala and make it available to bear writers so that
+they can write advanced linting bears. This will aim at supporting the more
+flexible visitor based method of AST traversal (as opposed to listener based
+mechanisms).
+
+The proposal is to introduce a parallel concept to the coala-bears library,
+which will be called the coala-ast hereon.
+
+## Proposed Change
+
+Here is the detailed implementation stepwise:
+
+````
+1. There will be a separate repository named as coala-antlr-ast which will
+   be installable via ```pip install coala-antlr-ast```.
+2. A new package would be introduced in the coala-antlr-ast repository,
+   which would be the maintain all the dependencies for antlr-ast based
+   bears. This would be the ```antlrastlib``` package
+3. The ```antlrastlib``` package will provide endpoints relevant to the
+   visitor model of traversing the ast generated via ANTLR.
+4. Another package named ```antlrparsers``` will be created inside
+   coala-antlr-ast repository. This package will hold parsers generated
+   for various languages generated (with python target) beforehand for the
+   supported set of grammars.
+5. ```antlrastlib``` will have an AST loader class that will be responsible
+   for loading the AST of a given file depending on the extension.
+   For e.g, a .c file will be loaded using the parser from
+   ```antlrastlib``` for c language.
+6. Another AST walker class will be responsible for providing endpoints for
+   traversing the AST using methods such as ```get_next_node``` and
+   ```get_previous_node```.
+````
+
+## Management of the new `coala-ast` repository
+
+Managing a new repository is a heavy task, and this will be highly automated as
+follows:
+
+```
+1. The parsers in coala-antlr-ast/antlrparsers would be generated and
+   pushed via travis-builds whenever a new commit is pushed.
+2. The cib tool can be enhanced to deal with the installation of bears that
+   require only some specified parsers (for e.g a PyUnusedVarBear would
+   only require parser for python)
+3. The cib tool can also trigger specialised builds and download the newly
+   generated parser on the fly
+```
+
+## Code Samples/Prototypes
+
+Here is a prototype for the implementations within antlrastlib:
+
+```python
+import antlrparsers
+import antlr4
+
+class ASTLoader():
+    lexerMapping = {
+        'c' : antlrparsers.clexer,
+        'py' : antlrparsers.pylexer,
+        'js' : antlrparsers.jslexer,
+        ...
+    }
+    parserMapping = {
+        'c' : antlrparsers.cparser,
+        'py' : antlrparsers.pyparser,
+        'js' : antlrparsers.jsparser,
+        ...
+    }
+
+    @staticmethod
+    def loadFile(file,filename):
+        """ Loads file's AST into memory """
+        ext = get_extension(filename)
+        if ext is None:
+            raise FileExtensionNotSupported
+        inputStream = file.readlines()
+        lexer = lexerMapping[ext](inputStream)
+        parser = parserMapping[ext](lexer)
+        return parser
+
+class ASTWalker():
+    treeRoot = None
+    tree = None
+    def __init__(file,filename):
+        self.tree = ASTLoader.load(file, filename)
+        self.treeRoot = self.tree
+
+    def get_next_node(self):
+        """ Modify the internal tree variable and return the next node """
+
+    def get_prev_node(self):
+        """ Modify the internal tree variable and return prev node """
+
+    def reset(self):
+        """ Reset the internal tree variable to tree root """
+
+    def isTerminalNode(node):
+        """ Determine if the node is terminal """
+        return isinstance(node, antlr4.tree.Tree.TerminalNode)
+
+    ...
+```
+
+### Prototype of  `ASTBear` class implementation:
+
+```python
+from coalib.bears.LocalBear import Bear
+from antlrastlib import ASTWalker
+
+class ASTBear(LocalBear):
+    walker = None
+
+    def initialise(file, filename):
+        walker = ASTWalker(file, filename)
+
+    def run(self,
+            filename,
+            file,
+            tree,
+            *args,
+            dependency_results=None,
+            **kwargs):
+        raise NotImplementedError           # Needs to be done by bear
+```
+
+### A test bear:
+
+```python
+from coalib.results.Result import Result
+from antrlastlib.ASTBear import ASTBear
+
+class TestBear(ASTBear):
+    def run(self,
+            filename,
+            file,
+            tree,
+            *args,
+            dependency_results=None,
+            **kwargs):
+        self.initialise(file, filename)
+        num_nodes = 0
+        while(tree.get_next_node() != None):
+            num_nodes += 1
+        yield Result(self, 'File {} has {} nodes '
+                           ' in AST.'.format(filename, num_nodes))
+```