---
title: "What We’re Reading: Single-Queue Decoding for Neural Machine Translation"
description: The Lilt team's review of a paper by Raphael Shu and Hideki Nakayama. They found that by storing all hypotheses in a single priority queue and using a universal score function for hypothesis selection, they could improve upon the quality of hypotheses in the search space.
---

[Lilt Labs | Bridging Translation Research and Practice](https://labs.lilt.com)

# [What We’re Reading: Single-Queue Decoding for Neural Machine Translation](https://labs.lilt.com/what-were-reading-single-queue-decoding-for-neural-machine-translation)

 Written by [Spence Green](https://labs.lilt.com/author/spence-green) | August, 2, 2017

The most popular way of finding a translation for a source sentence with a neural sequence-to-sequence model is a simple beam search. The target sentence is predicted one word at a time and after each prediction, a fixed number of possibilities (typically between 4 and 10) is retained for further exploration. This strategy can be suboptimal as these local hard decisions do not take the remainder of the translation into account and can not be reverted later on.

In this paper, the authors propose to tackle this problem by keeping a single stack of partial hypotheses with different lengths alive. If a previous decision turns out to be sub-optimal a few words later, it is now possible to revisit them. To make the determination the authors use an additional tiny neural network to predict the expected sentence length, for each source sentence and each partial translation. If the two differ significantly it is considered an indicator that a previous decision should be revisited.

Paper: **[Single-Queue Decoding for Neural Machine Translation](https://arxiv.org/abs/1707.01830)**  
Authors: Raphael Shu, Hideki Nakayama  
Venue: University of Tokyo

[View full post](https://labs.lilt.com/what-were-reading-single-queue-decoding-for-neural-machine-translation)

```json
{
  "@context" : "http://schema.org",
  "@type" : "BlogPosting",
  "author" : {
    "@type" : "Person",
    "name" : "Spence Green"
  },
  "dateModified" : "2018-06-27T18:32:43.229Z",
  "datePublished" : "2017-08-02T11:15:00Z",
  "headline" : "What We’re Reading: Single-Queue Decoding for Neural Machine Translation",
  "image" : {
    "@type" : "ImageObject",
    "height" : 60,
    "url" : "/hs/hsstatic/content_shared_assets/static-1.4092/img/default-amp-logo.png",
    "width" : 60
  },
  "mainEntityOfPage" : "https://labs.lilt.com/what-were-reading-single-queue-decoding-for-neural-machine-translation",
  "publisher" : {
    "@type" : "Organization",
    "logo" : {
      "@type" : "ImageObject",
      "height" : 60,
      "url" : "/hs/hsstatic/content_shared_assets/static-1.4092/img/default-amp-logo.png",
      "width" : 60
    },
    "name" : "Lilt Labs"
  }
}
```