Images courtesy Samsung.

Samsung Research, the advanced R&D hub of Samsung Electronics’ SET (end-products) business, has ranked first in two of the world’s top global artificial intelligence (AI) machine reading comprehension competitions.

Samsung Research recently placed first in the Machine Reading Comprehension (MS MARCO) Competition held by Microsoft (MS), as well as showing the best performance in TriviaQA* hosted by the University of Washington, proving the excellence of its AI algorithm.

With intense competition in developing AI technologies globally, machine reading comprehension competitions such as MS MARCO are booming around the world. MS MARCO and TriviaQA are among the actively researched and used machine reading comprehension competitions along with SQuAD of Stanford University and NarrativeQA of DeepMind. Distinguished universities around the world and global AI firms including Samsung are competing in these challenges.

Machine reading comprehension is where an AI algorithm is tasked with analyzing data and finding an optimum answer to a query on its own accord. For MS MARCO and TriviaQA, AI algorithms are tested in their capabilities of processing natural language in human Q&As and also providing written text in various types of documents such as news articles and blog posts.

For example in MS MARCO, ten web documents are presented for a certain query to let an AI algorithm create an optimum answer. Queries are randomly selected from a million queries from Bing (MS search engine) users. Answers are evaluated statistically by estimating how close they are with human answers. This is a test designed to apply an AI algorithm to solve real-world problems.

Samsung Research took part in the competitions with ConZNet, an AI algorithm developed by the company’s AI Center. ConZNet features skillful capabilities through adopting the Reinforcement Learning** technique, which advances machine intelligence by giving reasonable feedback for outcomes, similar to a stick-and-carrot strategy in a learning process.

With the recent acceleration in global competition to develop AI technologies, contests are widespread in areas of computer vision (technologies to analyze characters and images) and visual Q&A to solve problems using recognized images of characters as well as machine reading comprehension.

The Beijing branch of Samsung Research won the International Conference on Document Analysis and Recognition (ICDAR) hosted by the International Association of Pattern Recognition (IAPR) in March, putting them in a top-tier group for global computer vision tests. The ICDAR is the most influential competition in Optical Character Recognition (OCR) technologies.

“We are developing an AI algorithm to provide answers to user queries in a simpler and more convenient manner, for real life purposes,” said Jihie Kim, Head of Language Understanding Lab at Samsung Research. “Active discussion is underway in Samsung to adopt the ConZNet AI algorithm for products, services, customer response and technological development.”

An interview with Jihie Kim, Head of Language Understanding Lab, Samsung Research

The question of how AI technologies understand human dialog and queries to suggest an optimum answer is one of the hot topics in the AI industry. Jihie Kim, Head of the Language Understanding Lab at Samsung Research AI Center, is also striving to develop the technology behind an AI algorithm that can talk with people naturally and propose solutions to a problem.

The Language Understanding Lab led by Dr. Kim recently grabbed global attention after placing top ranks at global machine reading comprehension competitions held by Microsoft and the University of Washington, respectively. Samsung Newsroom visited the Samsung Research AI center in Seocho-gu, Korea to interview Dr. Kim about AI performance in the machine reading comprehension competitions and a future evolution plan for AI algorithms.

An interview with Jihie Kim, Head of Language Understanding Lab, Samsung Research

Jihie Kim, Head of the Language Understanding Lab at Samsung Research

Q. Please tell us about the MS MARCO and TriviaQA competitions held by Microsoft and the University of Washington, respectively, where your team ranked first place.

A. There have been many global machine reading competitions recently where AI presents solutions to a problem. MS MARCO and TriviaQA are among the top five global competitions in machine reading comprehension. AI algorithms are tested on whether they can understand and analyze questions to offer answers. Those tests are designed by referring to internet users’ queries and search results.

Q. What do you think was the critical factor in excelling at the AI competitions which require such high levels of technical expertise?

A. The ConZNet algorithm developed by the Language Understanding Lab at Samsung Reseach is upgrading its intelligence by considering real user environments. The algorithm takes natural language into account such as how people deliver queries and answers online. We were able to win those competitions because the MS MARCO and TriviaQA competitions are about AI capabilities in real user environments. In truth, our algorithm was a bit behind other competitors in tests requiring a simple answer to a question after analyzing a short paragraph. But because such technologies have low relevance to real environments using AI technologies, we are focusing on the other tests such as MS MARCO in proceeding with continuous R&D.

Q. Do you apply the winning algorithms to customer services in real life?

A. An Open Lab event was held recently to introduce the labs at Samsung Research to other departments in Samsung Electronics. At the event, we had in-depth discussions with engineers in our home appliances and smartphone departments about AI algorithms. Departments dealing with customer services also showed high interest in what we do because AI-based customer services including chatbots are emerging as a hot topic. We hope that our technologies developed at Samsung Research will be naturally adopted to Samsung Electronics products and services.

Q. What is your future evolution plan for advancing AI technologies in language understanding?

A. ConZNet is an acronym for “Context Zoom-in Network.” The name implies that understanding the context of what people say is critical. We need to advance AI technologies to help them understand and analyze short sentences. AI algorithms also need to have capabilities to analyze real-time news reports rather than existing data to give answers to customer queries. We are also developing technologies where an AI algorithm can answer, “there are no proper answers to your query,” as well as search for right answers. The so-called “rejection problem” is an AI technology with a high level of technical difficulties.

Q. Please tell us your ultimate goal in developing AI technologies.

A. The strengths of Samsung in the AI industry are that we can build a knowledge system about connections between machines and applications, and customer demands in the internet of things (IoT) environment comprised of personal devices, based on Samsung Electronics’ diverse product lineup. This will help us to achive the goal of realizing a user-oriented AI system by collaborating with global partners in the industry. Samsung Electronics recently began to launch global AI Centers and we will lead the effort of working with AI experts at the new centers abroad.

* Competitions such as MS MARCO and TriviaQA allow contestants to participate at any time, and rankings are altered according to real-time test results.

** The Reinforcement Learning is the most advanced Machine Learning AI algorithm, and cutting-edge AI technologies including AlphaGo are upgrading machine intelligence by applying this technique.