Crashes when files contain the ESC symbol (`\x1b`, e.g. an unescaped ANSI sequence)

[ilyagr]

I see a 500 error in my browser after clicking on a button to go tho such a file's diff, and this error printed in the console:

javax.xml.stream.XMLStreamException: ParseError at [row,col]:[298,5]
Message: An invalid XML character (Unicode: 0x1b) was found in the value of attribute "label" and element is "tree".
	at java.xml/com.sun.org.apache.xerces.internal.impl.XMLStreamReaderImpl.next(XMLStreamReaderImpl.java:652)
	at java.xml/com.sun.xml.internal.stream.XMLEventReaderImpl.nextEvent(XMLEventReaderImpl.java:83)
	at com.github.gumtreediff.io.TreeIoUtils$XmlInternalGenerator.generate(TreeIoUtils.java:783)
	at com.github.gumtreediff.gen.TreeGenerator.generateTree(TreeGenerator.java:41)
	at com.github.gumtreediff.gen.TreeGenerator$ReaderConfigurator.reader(TreeGenerator.java:119)
	at com.github.gumtreediff.gen.TreeGenerator$ReaderConfigurator.string(TreeGenerator.java:135)
	at com.github.gumtreediff.gen.treesitter.AbstractTreeSitterGenerator.generate(AbstractTreeSitterGenerator.java:39)
	at com.github.gumtreediff.gen.TreeGenerator.generateTree(TreeGenerator.java:41)
	at com.github.gumtreediff.gen.TreeGenerator$ReaderConfigurator.reader(TreeGenerator.java:119)
	at com.github.gumtreediff.gen.TreeGenerator$ReaderConfigurator.file(TreeGenerator.java:90)
	at com.github.gumtreediff.gen.TreeGenerator$ReaderConfigurator.file(TreeGenerator.java:100)
	at com.github.gumtreediff.gen.TreeGenerators.getTree(TreeGenerators.java:58)
	at com.github.gumtreediff.gen.TreeGenerators.getTree(TreeGenerators.java:70)
	at com.github.gumtreediff.actions.Diff.compute(Diff.java:81)
	at com.github.gumtreediff.client.diff.AbstractDiffClient.getDiff(AbstractDiffClient.java:132)
	at com.github.gumtreediff.client.diff.webdiff.WebDiff.lambda$configureSpark$6(WebDiff.java:123)
	at spark.RouteImpl$1.handle(RouteImpl.java:72)
	at spark.http.matching.Routes.execute(Routes.java:61)
	at spark.http.matching.MatcherFilter.doFilter(MatcherFilter.java:134)
	at spark.embeddedserver.jetty.JettyHandler.doHandle(JettyHandler.java:50)
	at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1598)
	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
	at org.eclipse.jetty.server.Server.handle(Server.java:516)
	at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487)
	at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732)
	at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479)
	at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277)
	at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311)
	at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105)
	at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104)
	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338)
	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315)
	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173)
	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131)
	at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409)
	at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883)
	at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034)
	at java.base/java.lang.Thread.run(Thread.java:840)

I'm not currently sure whether this affects all diff methods or just some. In this case, the ESC symbol was part of a Rust string inside the file full of tests (Rust strings allow any Unicode character)

Here is an example of a file that causes the error. To reproduce the bug, you can, for example, delete a line from the comment in the beginning, and try to compare the old and the new version with gumtree.

https://github.com/jj-vcs/jj/blob/5666f4ac23b88364fb4290cf433a24f3ab20d1d7/cli/tests/test_global_opts.rs

[jrfaller]

Hi and thanks for the report. Do you happen to know which generator was used to parse the Rust code? It seems to be the external tree-sitter-parser in Python, right?

[ilyagr]

I'm unsure, I just used the docker image.

[jrfaller]

OK therefore should be the python bridge to tree-sitter. This bridge goes through a XML serialization / deserialization phase. Apparently the problem comes from the XML parser (javax.xml) that does not handle the \x1b character. I don't know if it's fixable. We have now a native tree-sitter bridge in the latest GumTree's version, maybe you could try that? Cheers.