Challenges In Automated Review Analysis

Determining sentiment in the text using neural network models has become almost a school task for beginner data scientists. However, despite the high quality of models and the ease of working with them, it is not so easy to characterize real data from reviews of restaurants, hotels, or museums. Even Google’s algorithms can’t do it properly.

What are the reasons?

In some reviews, the text contains a lot of neutral information, from which it is difficult to automatically highlight something specific:

Sometimes, even if the review is unequivocally positive, it is difficult to understand what exactly attracted users if there are many different topics raised in one review.

Searching only by keywords does not always help, as clients call the same things differently, for example, “intimate place” and “relaxed atmosphere” is about the same thing.

Analyzing each sentence individually does not often help if the punctuation is not observed or a sentence contains several sentiments.

Examples, please.

Let’s take a closer look at this review.

We understand very clearly that a customer is happy about coffee and disappointed by the service, despite the smiling emoji in the end. But for artificial neural networks, everything is not so clear.

Here is the result of Google’s sentiment analysis from Natural Language API (Google Service).

Sentiment 0 (no answer) about “service”, and very low positive sentiment about “coffee”. Not enough, right?

For example, our sentiment analysis for this review is

“The coffee is delicious, but the service is a tragedy :)” — 0.994435399093432 [coffee]

“The coffee is delicious, but the service is a tragedy :)” — 0.012208339626454045 [service]

Where 1 — is excellent, 0 — is terrible

Much better, agree?

Our developed method is based on the autoencoder architecture of the neural networks, it allows ‘extracting’ all the mentioned objects and the sentiment related specifically to them from the review.

For example:

Our algorithm ‘understood’ that a customer is angry about both the barista and the service. It has never been possible to automatically reach such a conclusion.

It shows our model copes with rather complicated cases when one word can have different meanings depending on the context.

In these reviews:

Our model rated coffee low (0.03).

But in these cases

“cold coffee” has got a high rating (0.99).

In this example, the model determined the problem was “cold coffee” not just coffee.

Mark for cold coffee — is 0.02

As you can see the model copes with complex cases.

Here is a small demo that shows the rating of the cafe in the context of different criteria.

We get data about all the chain cafes in Vilnius.

You can choose 4 topics you are interested in and they will be shown on the map.

When you choose an object, all reviews and the ability to filter by topic will be shown.

Certainly, there are much more ways how to visualize data and show more parameters, for example here is the spider diagram for a UK chain of Italian restaurants.

It is possible to track new reviews in real-time, identify problem areas in the chain and respond to them in a timely manner.

Do you want to see your customers' feedback?

Reach out to us and our AI model will analyze them.