Java或Python用于自然语言处理

问题:Java或Python用于自然语言处理

我想知道哪种编程语言更适合自然语言处理。Java还是Python?我发现了很多与此有关的问题和答案。但是我仍然迷失在选择使用哪一个上。

我想知道用于Java的NLP库,因为有很多库(LingPipe,GATE,OpenNLP,StandfordNLP)。对于Python,大多数程序员都建议使用NLTK。

但是,如果我要对非结构化数据(只是自由格式的纯英文文本)进行一些文本处理或信息提取,以获得一些有用的信息,那么最佳选择是什么?Java还是Python?合适的图书馆?

更新

我要做的是从非结构化数据中提取有用的产品信息(例如,用户使用不太标准的英语来制作有关手机或笔记本电脑的不同形式的广告)

I would like to know which programming language is better for natural language processing. Java or Python? I have found lots of questions and answers regarding about it. But I am still lost in choosing which one to use.

And I want to know which NLP library to use for Java since there are lots of libraries (LingPipe, GATE, OpenNLP, StandfordNLP). For Python, most programmers recommend NLTK.

But if I am to do some text processing or information extraction from unstructured data (just free formed plain English text) to get some useful information, what is the best option? Java or Python? Suitable library?

Updated

What I want to do is to extract useful product information from unstructured data (E.g. users make different forms of advertisement about mobiles or laptops with not very standard English language)


回答 0

Java vs Python for NLP非常偏爱或必需。根据公司/项目的不同,您将需要使用其中一个,而除非您负责一个项目,否则通常没有太多选择。

除了NLTK(www.nltk.org),实际上还有其他用于文本处理的库python

(有关更多信息,请参见https://pypi.python.org/pypi?%3Aaction=search&term=natural+language+processing&submit=search

对于Java,还有其他许多吨,但这是另一个清单:

这是基本字符串处理的不错比较,请参阅http://nltk.googlecode.com/svn/trunk/doc/howto/nlp-python.html

GATE与UIMA与OpenNLP的有用比较,请参阅https://www.assembla.com/spaces/extraction-of-cost-data/wiki/Gate-vs-UIMA-vs-OpenNLP?version=4

如果您不确定使用NLP的语言是什么,我个人会说“可以为您提供所需分析/输出的任何语言”,请参阅要学习自然语言处理的语言或工具?

这是NLP工具的最新版本(2017):https : //github.com/alvations/awesome-community-curated-nlp

NLP工具的较旧列表(2013):http ://web.archive.org/web/20130703190201/http: //yauhenklimovich.wordpress.com/2013/05/20/tools-nlp


除了语言处理工具之外,您非常需要将machine learning工具合并到NLP管道中。

有一个整体的范围PythonJava,并再次就看个人喜好和库是否人性化不够:

python中的机器学习库:

(有关更多信息,请参见https://pypi.python.org/pypi?%3Aaction=search&term=machine+learning&submit=search


随着最近(2015年)NLP中的深度学习海啸,您可能可以考虑:https : //en.wikipedia.org/wiki/Comparison_of_deep_learning_software

我将避免出于非偏爱/中立的目的列出深度学习工具。


其他也需要NLP / ML工具的Stackoverflow问题:

Java vs Python for NLP is very much a preference or necessity. Depending on the company/projects you’ll need to use one or the other and often there isn’t much of a choice unless you’re heading a project.

Other than NLTK (www.nltk.org), there are actually other libraries for text processing in python:

(for more, see https://pypi.python.org/pypi?%3Aaction=search&term=natural+language+processing&submit=search)

For Java, there’re tonnes of others but here’s another list:

This is a nice comparison for basic string processing, see http://nltk.googlecode.com/svn/trunk/doc/howto/nlp-python.html

A useful comparison of GATE vs UIMA vs OpenNLP, see https://www.assembla.com/spaces/extraction-of-cost-data/wiki/Gate-vs-UIMA-vs-OpenNLP?version=4

If you’re uncertain, which is the language to go for NLP, personally i say, “any language that will give you the desired analysis/output”, see Which language or tools to learn for natural language processing?

Here’s a pretty recent (2017) of NLP tools: https://github.com/alvations/awesome-community-curated-nlp

An older list of NLP tools (2013): http://web.archive.org/web/20130703190201/http://yauhenklimovich.wordpress.com/2013/05/20/tools-nlp


Other than language processing tools, you would very much need machine learning tools to incorporate into NLP pipelines.

There’s a whole range in Python and Java, and once again it’s up to preference and whether the libraries are user-friendly enough:

Machine Learning libraries in python:

(for more, see https://pypi.python.org/pypi?%3Aaction=search&term=machine+learning&submit=search)


With the recent (2015) deep learning tsunami in NLP, possibly you could consider: https://en.wikipedia.org/wiki/Comparison_of_deep_learning_software

I’ll avoid listing deep learning tools out of non-favoritism / neutrality.


Other Stackoverflow questions that also asked for NLP/ML tools:


回答 1

这个问题很开放。就是说,下面而不是选择一个,而是根据您要使用的语言进行比较(因为两种语言都有不错的库)。

Python

在Python方面,首先要看的是Python Natural Language Toolkit。正如他们在描述中所指出的那样,NLTK是构建Python程序以使用人类语言数据的领先平台。它为50多种语料库和词汇资源(如WordNet)提供了易于使用的界面,并提供了一套用于分类,标记化,词干,标记,解析和语义推理的文本处理库。

您还可以查找一些出色的代码,这些代码源自基于Python的Google自然语言工具包项目。您可以在GitHub上找到该代码的链接。

爪哇

首先看的是斯坦福大学的自然语言处理小组。那里分发的所有软件都是用Java编写的。所有最新发行版都需要Oracle Java 6+或OpenJDK 7+。分发程序包包括用于命令行调用的组件,jar文件,Java API和源代码。

您在许多机器学习环境中看到的另一个很棒的选择(通用选择)是Weka。Weka是用于数据挖掘任务的机器学习算法的集合。这些算法既可以直接应用于数据集,也可以从您自己的Java代码中调用。Weka包含用于数据预处理,分类,回归,聚类,关联规则和可视化的工具。它也非常适合开发新的机器学习方案。

The question is very open ended. That said, rather than choose one, below is a comparison depending on the language that you would like to use (since there are good libraries available in both languages).

Python

In terms of Python, the first place you should look at is the Python Natural Language Toolkit. As they note in their description, NLTK is a leading platform for building Python programs to work with human language data. It provides easy-to-use interfaces to over 50 corpora and lexical resources such as WordNet, along with a suite of text processing libraries for classification, tokenization, stemming, tagging, parsing, and semantic reasoning.

There is also some excellent code that you can look up that originated out of Google’s Natural Language Toolkit project that is Python based. You can find a link to that code here on GitHub.

Java

The first place to look would be Stanford’s Natural Language Processing Group. All of software that is distributed there is written in Java. All recent distributions require Oracle Java 6+ or OpenJDK 7+. Distribution packages include components for command-line invocation, jar files, a Java API, and source code.

Another great option that you see in a lot of machine learning environments here (general option), is Weka. Weka is a collection of machine learning algorithms for data mining tasks. The algorithms can either be applied directly to a dataset or called from your own Java code. Weka contains tools for data pre-processing, classification, regression, clustering, association rules, and visualization. It is also well-suited for developing new machine learning schemes.