JRuby support using jieba-analysis library for Java

[headius]

cppjieba_rb is a dependency of Discourse, which I am trying to get to run on JRuby. However, JRuby does not support CRuby extensions.

There is a Java library (https://github.com/huaban/jieba-analysis) that could be used from JRuby to provide the same functionality (or at least, I think so...I am not familiar with the jieba library and English documentation online is rather sparse).

Perhaps we can work together to get a jieba library for Ruby that works with both the native extension and the JVM library?

[erickguan]

Do you prefer to replace this gem with another?

Perhaps we can work together to get a jieba library for Ruby that works with both the native extension and the JVM library?

I don't have any experience about that. But sure, we can work on this together and provide you necessary information.

[headius]

The library I pointed at can be used from Ruby directly in JRuby, so it should be possible to mimic the API you've provided in cppjieba without writing any Java code at all.

[erickguan]

It's essentially the same algorithm but with different API.

So they provided with two mode here. We can patch this. For example, :mix can be mapped to SEARCH here. Others can be INDEX. (It doesn't make sense for NLP task we tried to perform)

https://github.com/huaban/jieba-analysis/blob/master/src/main/java/com/huaban/analysis/jieba/JiebaSegmenter.java#L15-L18

jieba-analysis has its own DICT_PATH and a customizable USER_DICT. I think we could change file path here.

Could you go for a PR with these information?