--- title: "What We’re Reading: Domain Attention with an Ensemble of Experts" description: This month, Lilt's team came across a paper outlining how to improve the performance of domain adaptation, which is often a major obstacle in effectively deploying machine learning systems. --- - [Lilt.com](https://lilt.com/) - [Resources](https://lilt.com/resources) - [Lilt.com](https://lilt.com/) - [Resources](https://lilt.com/resources) ## What We’re Reading: Domain Attention with an Ensemble of Experts ![](https://labs.lilt.com/hubfs/spence.jpeg) by Spence Green August, 15, 2017 1 Minute Read - - - A major problem in effective deployment of machine learning systems in practice is domain adaptation — given a large auxiliary supervised dataset and a smaller dataset of interest, using the auxiliary dataset to increase performance on the smaller dataset. This paper considers the case where we have K datasets from distinct domains and adapting quickly to a new dataset. It learns K separate models on each of the K datasets and treats each as experts. Then given a new domain it creates another model for this domain, but in addition, computes attention over the experts. It computes attention via a dot product that computes the similarity of the new domain’s hidden representation with the other K domains’ representations. In addition to this core idea, the authors propose a couple modifications to improve performance. In the first, they compute an additional form of an attention. This time they take the inner product between the new domain’s hidden layer and the label embedding of the most likely output of the expert networks. They also find that sparsely choosing experts by only keeping the top K’ experts non-zero improves performance. They choose K’ via grid search on the validation set. Paper: [Domain Attention with an Ensemble of Experts](http://www.karlstratos.com/publications/acl17ensemble.pdf) Authors: Young-Bum Kim, Karl Stratos, Dongchan Kim [View All Posts](https://labs.lilt.com) ### [![](https://labs.lilt.com/hubfs/Deck_%20Women%20in%20Loc%20Webinar%20%20(1).png) March, 2, 2023 WLSP: Maximize and Scale Your Localization Programs In 2023 Webinar Recap 6 Minute Read ![](https://labs.lilt.com/hubfs/elly.jpeg) Our recent webinar featured industry experts: • Tanya Gadea (VP of Production at Lilt) • Loy Searle (Senior Director of Localization & Globalization at Workday) • Brittney Benchoff (Head of Product at Lilt) In this blog post, we share what was discussed in this discussion and provide three key takeaways. Read More](https://labs.lilt.com/wlsp-maximize-and-scale-your-localization-programs-in-2023-webinar-recap?hsLang=en) ### [![](https://labs.lilt.com/hubfs/AI%20LLM%20Masterclass%20Pt%201%20-%20Featured%20Image.png) August, 18, 2023 Evaluating Risks and Trade-offs in an AI Strategy: Webinar Recap 6 Minute Read ![](https://labs.lilt.com/hubfs/elly.jpeg) AI has created new opportunities and challenges traditional enterprise work. In our latest webinar, Spence Green, CEO of Lilt, delved into the risks and trade-offs of investing in an AI strategy and how to leverage it to drive enterprise-level changes. Read More](https://labs.lilt.com/evaluating-risks-and-trade-offs-in-an-ai-strategy?hsLang=en) - [System Status](https://status.lilt.com/) - [Support Terms](https://lilt.com/terms) - [Privacy](https://lilt.com/privacy) - - - -  2026 Lilt, Inc.