{"id":717,"date":"2018-01-17T15:02:16","date_gmt":"2018-01-17T15:02:16","guid":{"rendered":"http:\/\/www.marekrei.com\/blog\/?p=717"},"modified":"2019-09-27T23:22:28","modified_gmt":"2019-09-27T23:22:28","slug":"paper-summaries","status":"publish","type":"post","link":"https:\/\/www.marekrei.com\/blog\/paper-summaries\/","title":{"rendered":"57 Summaries of Machine Learning and NLP Research"},"content":{"rendered":"

Staying on top of recent work is an important part of being a good researcher, but this can be quite difficult. Thousands of new papers are published every year at the main ML and NLP conferences<\/a>, not to mention all the specialised workshops and everything that shows up on ArXiv. Going through all of them, even just to find the papers that you want to read in more depth, can be very time-consuming.<\/p>\n

In this post, I have summarised 50 papers. After going through a paper, if I had the chance, I would write down a few notes and summarise the work in a couple of sentences. These are not meant as reviews – I’m not commenting on whether I think the paper is good or not. But I do try to present the crux of the paper as bluntly as possible, without unnecessary sales tactics. Hopefully this can give you the general idea of 50 papers, in roughly 20 minutes of reading time.<\/p>\n

The papers are not selected or ordered based on any criteria. It is not a list of the best papers I have read, more like a random sample. The only filter that I applied was to exclude papers older than 2016, as the goal is to give an overview of the more recent work.<\/p>\n

I set out to summarise 50 papers. Once I was done, I thought this would be a sensible place to summarise my own work as well. So at the end of the list you will also find brief summaries of the papers I published in 2017.<\/p>\n

Let’s get started.<\/p>\n

1. A Thorough Examination of the CNN\/Daily Mail Reading Comprehension Task<\/strong>
\nDanqi Chen, Jason Bolton, Christopher D. Manning. Stanford. ACL 2016.
\nhttps:\/\/arxiv.org\/pdf\/1606.02858.pdf<\/a><\/p>\n

Hermann et al (2015) created a dataset for testing reading comprehension by extracting summarised bullet points from CNN and Daily Mail. All the entities in the text are anonymised and the task is to place correct entities into empty slots\u00a0based on the news article.<\/p>\n

\"cnn_daily_mail\"<\/a><\/p>\n

This paper has hand-reviewed 100 samples from the dataset and concludes that around 25% of the questions are difficult or impossible to answer even for a human, mostly due to the anonymisation process. They present a simple classifier that achieves unexpectedly good results, and a neural network based on attention that beats all previous results by quite a margin.<\/p>\n

2. Word Translation Without Parallel Data<\/strong>
\nAlexis Conneau, Guillaume Lample, Marc’Aurelio Ranzato, Ludovic Denoyer, Herv\u00e9 J\u00e9gou. Facebook, Le Mans, Sorbonne. ArXiv 2017.
\nhttps:\/\/arxiv.org\/pdf\/1710.04087.pdf<\/a><\/p>\n

Inducing word translations using only monolingual corpora for two languages. Separate embeddings are trained for each language and a mapping is learned though an adversarial objective, along with an orthogonality constraint on the most frequent words. A strategy for an unsupervised stopping criterion is also proposed.<\/p>\n

\"Word<\/a><\/p>\n

<\/p>\n

3. A Nested Attention Neural Hybrid Model for Grammatical Error Correction<\/strong>
\nJianshu Ji, Qinlong Wang, Kristina Toutanova, Yongen Gong, Steven Truong, Jianfeng Gao. ACL 2017.
\nhttp:\/\/aclweb.org\/anthology\/P\/P17\/P17-1070.pdf<\/a><\/p>\n

Proposing character-based extensions to a neural MT system for grammatical error correction. OOV words are represented in the encoder and decoder using character-based RNNs. They evaluate on the CoNLL-14 dataset, integrate probabilities from a large language model, and achieve good results.<\/p>\n

\"A<\/a><\/p>\n

4. On-line Active Reward Learning for Policy Optimisation in Spoken Dialogue Systems
\n<\/strong>Pei-Hao\u00a0Su,\u00a0Milica\u00a0Gasic,\u00a0Nikola\u00a0Mrksic,\u00a0Lina\u00a0Rojas-Barahona,\u00a0Stefan\u00a0Ultes,\u00a0David\u00a0Vandyke,\u00a0Tsung-Hsien\u00a0Wen,\u00a0Steve\u00a0Young. Cambridge. ACL 2016.
\nhttp:\/\/aclweb.org\/anthology\/P\/P16\/P16-1230.pdf<\/a><\/p>\n

The goal\u00a0is to improve the training process for a spoken dialogue system, more specifically a telephone-based system providing restaurant information for the Cambridge (UK) area. They train a supervised system which tries to predict the success on the current dialogue – if the model is certain about the outcome, the predicted label is used for training the dialogue system; if the model is uncertain, the user is asked to provide a label. Essentially it reduces the amount of annotation that is required, by choosing which examples should be annotated through active learning.<\/p>\n

\"dialogue\"<\/a><\/p>\n

The dialogue is mapped to a vector representation using a bidirectional LSTM\u00a0trained like an autoencoder, and a Gaussian Process\u00a0is used for modelling dialogue success.<\/p>\n

5.\u00a0Vision and Feature Norms: Improving automatic feature norm learning through cross-modal maps<\/strong>
\nLuana Bulat, Douwe Kiela, Stephen Clark. Cambridge. NAACL 2016.
\nhttp:\/\/aclweb.org\/anthology\/N\/N16\/N16-1071.pdf<\/a><\/p>\n

The task is to predict feature norms – object properties, for example is_yellow<\/em> and is_edible<\/em> for the word\u00a0banana<\/em>. They experiment with adding in image recognition features, in addition to using distributional word vectors.<\/p>\n

An input word is used to retrieve 10 images from Google, these are passed through an ImageNet classifier to get feature vectors, and then averaged to get a vector representation for that word. A supervised\u00a0model (partial least-squares regression) is then trained to predict vectors of feature norms based on the input vectors (image-based, distributional, or a combination). Including the image information helps quite a bit, especially for detecting properties like colour and shape.<\/p>\n

\"Examples<\/a>
Examples of predicted feature norms using the visual features.<\/figcaption><\/figure>\n

6.\u00a0Adversarial examples in the physical world<\/strong>
\nAlexey Kurakin, Ian J. Goodfellow, Samy Bengio. Google, OpenAI. ArXiv.
\nhttps:\/\/arxiv.org\/abs\/1607.02533<\/a><\/p>\n

Adversarial examples are datapoints that are designed to fool a classifier. For example, we can take an image that is classified correctly using a neural network, then backprop through the model to find which changes we need to make in order for it to be classified as something else. And these changes can be quite small, such that a human would hardly notice a difference.<\/p>\n

\"Examples<\/a>
Examples of adversarial images.<\/figcaption><\/figure>\n

In this paper, they show that much of this property holds even when the images are fed into the classifier from the real world – after being photographed with a cell phone camera. While the accuracy goes from 85.3% to 36.3% when adversarial modifications are applied on the source images, the performance still drops from 79.8% to 36.4% when the images are photographed. They also propose two modifications to the process of generating adversarial images \u00a0– making it into a more gradual iterative process, and optimising for a specific adversarial class.<\/p>\n

7.\u00a0Extracting token-level signals of syntactic processing from fMRI – with an application to POS induction<\/strong>
\nJoachim\u00a0Bingel,\u00a0Maria Barrett,\u00a0Anders\u00a0S\u00f8gaard. Copenhagen. ACL 2016.
\nhttp:\/\/aclweb.org\/anthology\/P\/P16\/P16-1071.pdf<\/a><\/p>\n

They incorporate fMRI features into POS tagging, under the assumption that reading semantically\/functionally different words will activate the brain in different ways. For this they use a dataset of fMRI recordings, where the subjects were reading a chapter of Harry Potter. The main issue is that fMRI has very low temporal resolution – there is only one fMRI reading per 4 tokens, and\u00a0in general it takes around 4-14 seconds for something to show up in fMRI. Nevertheless, they construct token-level vectors by using a Gaussian weighted average, integrate them into an unsupervised POS tagger, and show that it is able to improve performance.<\/p>\n

\"Neural<\/a>
Neural activity by brain region, from Wehbe et al. (2014).<\/figcaption><\/figure>\n

 <\/p>\n

8. Joint Extraction of Events and Entities within a Document Context<\/strong>
\nBishan Yang, Tom\u00a0Mitchell. Carnegie Mellon. NAACL 2016.
\nhttp:\/\/aclweb.org\/anthology\/N\/N16\/N16-1033.pdf<\/a><\/p>\n

They propose a joint model for 1) identifying event keywords in a text, 2) identifying entities, and 3) identifying the connections between these events and entities. They also do this\u00a0across different sentences, jointly for the whole text.<\/p>\n

\"Example<\/a>
Example of the entity and event annotation that the system is modelling.<\/figcaption><\/figure>\n

The entity detection part is done with a CRF; the structure of an event is learned with a probabilistic graphical model; information is integrated\u00a0from surrounding sentences using a Stanford coreference system; and these are all tied together across the whole document using Integer Linear Programming.<\/p>\n

9.\u00a0Candidate re-ranking for SMT-based grammatical error correction<\/strong>
\nZheng Yuan, Ted\u00a0Briscoe,\u00a0Mariano Felice. Cambridge. BEA Workshop 2016.
\nhttps:\/\/www.aclweb.org\/anthology\/W\/W16\/W16-0530.pdf<\/a><\/p>\n

They improve\u00a0an existing error correction system by re-ranking its predictions. The basic approach uses machine translation to perform error correction on learner texts – the incorrect text is essentially translated into correct text. Here, they include a ranking SVM to score and reorder the n-best lists from the translation model.<\/p>\n

The reranking features include various internal scores from the translation model, the rank in the original ordering, language model probabilities trained on large corpora, language model scores based on only the n-best list, word-level translation probabilities, and sentence length features. They show improvement on two error correction datasets.<\/p>\n

\"Example<\/a>
Example output from the models.<\/figcaption><\/figure>\n

10.\u00a0Variational Neural Machine Translation<\/strong>
\nBiao Zhang, Deyi Xiong, Jinsong Su, Hong Duan, Min Zhang.\u00a0Soochow University,\u00a0Xiamen University. ArXiv.
\nhttps:\/\/arxiv.org\/abs\/1605.07869<\/a><\/p>\n

They start with the neural machine translation model using alignment, by Bahdanau et al. (2014), and add an extra variational component.<\/p>\n

\"vnmt\"<\/a><\/p>\n

The authors use two neural variational components to model a distribution over latent variables z that captures the semantics of a sentence being translated. First, they model the posterior probability of z, conditioned on both input and output. Then they also model the prior of z, conditioned only on the input. During training, these two distributions are optimised to be similar using Kullback-Leibler distance, and during testing the prior is used. They report improvements on Chinese-English and English-German translation, compared to\u00a0using the original encoder-decoder NMT framework.<\/p>\n

11.\u00a0Numerically Grounded Language Models for Semantic Error Correction<\/strong>
\nGeorgios P. Spithourakis,\u00a0Isabelle Augenstein,\u00a0Sebastian Riedel. UCL. EMNLP 2016.
\nhttps:\/\/arxiv.org\/abs\/1608.04147<\/a><\/p>\n

They create an LSTM neural language model that 1) has better handling of numerical values, and 2) is conditioned on a knowledge base.<\/p>\n

\"numerical_grounding\"<\/a><\/p>\n

First the the numerical value each token is given as an additional signal to the network at each time step. While we normally represent token “25” as a normal word embedding, we now also have an extra feature with numerical value float(25).\u00a0Second, they condition the language model on text in a knowledge base. All the information in the KB is converted to a string, passed through an LSTM and then used to condition the main LM.<\/p>\n

They evaluate on a dataset of 16,003 clinical records which come paired with small KB tuples of 20 possible attributes. The numerical grounding helps quite a bit, and the best results are obtained when the KB conditioning is also added.<\/p>\n

12.\u00a0Black Holes and White Rabbits : Metaphor Identification with Visual Features<\/strong>
\nEkaterina Shutova,\u00a0Douwe Kiela,\u00a0Jean Maillard. Cambridge. NAACL 2016.
\nhttps:\/\/www.aclweb.org\/anthology\/N\/N16\/N16-1020.pdf<\/a><\/p>\n

They build a system for detecting metaphors (“blind alley”, “honest meal”, etc) from literal word pairs.<\/p>\n

\"Annotated<\/a>
Annotated metaphor examples from Tsvetkov et al. (2014), used in this work.<\/figcaption><\/figure>\n

The basic system uses word embedding similarity – cosine between the word embeddings. Then they explore variations using phrase embeddings, cos(phrase-word2, word2), which is similar to the operations with word regularities by Mikolov.<\/p>\n

Finally, they create vector representations for words and phrases using visual information. The words are used as queries in Google Image Search, and the returned images are passed through an image detection network in order to obtain vector representations. The best final system performs the task separately using linguistic and visual vectors, and then combines the resulting scores.<\/p>\n

13.\u00a0Counter-fitting Word Vectors to Linguistic Constraints<\/strong>
\nNikola Mrk\u0161i\u0107,\u00a0Diarmuid \u00d3 S\u00e9aghdha,\u00a0Blaise Thomson,\u00a0Milica Ga\u0161i\u0107,\u00a0Lina Rojas-Barahona,\u00a0Pei-Hao Su,\u00a0David Vandyke,\u00a0Tsung-Hsien Wen,\u00a0Steve Young. Cambridge, Apple. NAACL 2016.
\nhttp:\/\/www.aclweb.org\/anthology\/N16-1018<\/a><\/p>\n

They describe a method for augmenting existing word embeddings with knowledge of semantic constraints. The idea is similar to retrofitting by Faruqui et al. (2015), but using additional constraints and a different optimisation function.<\/p>\n

\"1603-00892v1\"<\/a>
\nExisting word vectors are further optimised to 1) have high similarity for known synonyms, 2) have low similarity for known antonyms, and 3) have high similarity to words that were highly similar in the original space. They evaluate on SimLex-999, showing state-of-the-art performance. Also, they use the method to improve a dialogue tracking system.<\/p>\n

14.\u00a0Bidirectional RNN for Medical Event Detection in Electronic Health Records<\/strong>
\nAbhyuday N. Jagannatha,\u00a0Hong Yu. University of Massachusetts. NAACL 2016.
\nhttps:\/\/www.aclweb.org\/anthology\/N\/N16\/N16-1056.pdf<\/a><\/p>\n

The authors have a dataset of 780 electronic health records and they use it to detect various medical events such as adverse drug events, drug dosage, etc. The task is done by assigning a label to each word in the document.<\/p>\n

\"Annotation<\/a>
Annotation statistics for the corpus of health records.<\/figcaption><\/figure>\n

They look at CRFs, LSTMs and GRUs. Both LSTMs and GRUs outperform the CRF, but the best performance is achieved by a GRU trained on whole documents.<\/p>\n

15.\u00a0Symmetric Patterns and Coordinations: Fast and Enhanced Representations of Verbs and Adjectives<\/strong>
\nRoy Schwartz,\u00a0Roi Reichart,\u00a0Ari Rappoport.\u00a0The Hebrew Universit, IIT. NAACL 2016.
\nhttp:\/\/www.aclweb.org\/anthology\/N16-1060<\/a><\/p>\n

They train word2vec skip-gram embeddings using coordinations as context. They use 11 manual patterns to extract coordinations (eg “X and Y”, “either X or Y”, etc). From “boats or planes”, “boats” will be a context of “planes” and “planes” will be a context of “boats”.<\/p>\n

They evaluate on SimLex-999 and find that this performs badly on nouns. However, it beats normal skip-gram and dependency-based skip-gram on verbs and adjectives.<\/p>\n

16.\u00a0Comparing Data Sources and Architectures for Deep Visual Representation Learning in Semantics<\/strong>
\nDouwe Kiela,\u00a0Anita L. Ver\u0151,\u00a0Stephen Clark. Cambridge. EMNLP 2016.
\nhttps:\/\/aclweb.org\/anthology\/D\/D16\/D16-1043.pdf
\n<\/a><\/p>\n

The authors compare different image recognition models and image data sources for multimodal word representation learning.<\/p>\n

\"d16-1043\"<\/a>
Image recognition models used for vector generation<\/figcaption><\/figure>\n

Experiments are performed on SimLex-999 (similarity) and MEN (relatedness). The performance of different models (AlexNet, GoogLeNet, VGGNet) is found to be quite similar, with VGGNet performing slightly better at the cost of requiring more computation. Using search engines for image sources gives good coverage; ImageNet performs quite well with VGGNet; ESP Game dataset gave the lowest performance. Combining visual and linguistic vectors was found to be beneficial on both English and Italian.<\/p>\n

17.\u00a0Named Entity Recognition for Novel Types by Transfer Learning<\/strong>
\nLizhen Qu,\u00a0Gabriela Ferraro,\u00a0Liyuan Zhou,\u00a0Weiwei Hou,\u00a0Timothy Baldwin. Melbourne. EMNLP 2016.
\nhttp:\/\/aclweb.org\/anthology\/D16-1087<\/a><\/p>\n

The authors tackle the problem of domain adaptation for NER, where the label set of the target domain is different from the source domain.<\/p>\n

They first train a CRF model on the source domain. Next, they train a LR classifier to predict labels in the target domain, based on predicted label scores from the model. Finally, the weights from the classifier are used to initialise another CRF model, which is then fine-tuned on the target domain data.<\/p>\n

\"Performance<\/a>
Performance of the proposed model (TransInit) compared to baselines<\/figcaption><\/figure>\n

18.\u00a0Hybrid computing using a neural network with dynamic external memory<\/strong>
\nAlex Graves,\u00a0Greg Wayne,\u00a0Malcolm Reynolds et al. DeepMind. Nature.
\nhttp:\/\/www.nature.com\/nature\/journal\/v538\/n7626\/full\/nature20101.html<\/a><\/p>\n

The DeepMind guys present an extension to the Neural Turing Machine architecture.<\/p>\n

\"nature20101\"<\/a><\/p>\n

They call it a Differentiable Neural Computer (DNC) and it uses 1) an attention mechanism to access information in a matrix that acts as a memory, 2) an attention mechanism to save information to that memory, and 3) a transition matrix that stores information about the order in which rows in the memory are modified, in order to better handle sequential information. They test on the bAbI question answering dataset, a graph inference task, and on solving a puzzle of arranging blocks.<\/p>\n

19.\u00a0A Neural Approach to Automated Essay Scoring<\/strong>
\nKaveh Taghipour,\u00a0Hwee Tou Ng. Singapore. EMNLP 2016.
\nhttps:\/\/aclweb.org\/anthology\/D\/D16\/D16-1193.pdf<\/a><\/p>\n

The authors construct a neural network for automated essay scoring.<\/p>\n

\"d16-1193\"<\/a><\/p>\n

Convolution window of 3 is passed over the text, which is used as input to an LSTM. The output of the LSTM is averaged over all timesteps and then a single value in the range of [0,1] is predicted as a scaled-down score for the essay. They evaluate by measuring quadratic weighted Kappa on the Kaggle essay scoring dataset.<\/p>\n

20.\u00a0Globally Coherent Text Generation with Neural Checklist Models<\/strong>
\nChloe Kiddon,\u00a0Luke Zettlemoyer,\u00a0Yejin Choi. Washington. EMNLP 2016.
\nhttps:\/\/aclweb.org\/anthology\/D\/D16\/D16-1032.pdf<\/a><\/p>\n

They describe a neural model for text generation, which keeps track of a checklist of items that need to be mentioned in the text.<\/p>\n

\"\"<\/a>
\nThe basic system is an encoder-decoder GRU model for text generation. On top of that, the model uses attention over items that need to be mentioned and items that have already been mentioned, both of which are encoded as vectors. An additional cost objective encourages the checklist to be\u00a0filled by the end of the text. Evaluation is performed on recipe and dialogue generation.<\/p>\n

21.\u00a0Automatic Features for Essay Scoring \u2013 An Empirical Study<\/strong>
\nFei Dong,\u00a0Yue Zhang. Singapore. EMNLP 2016.
\nhttps:\/\/www.aclweb.org\/anthology\/D\/D16\/D16-1115.pdf<\/a><\/p>\n

The authors investigate convolutional networks for essay scoring. They use a two-level convolution – first over words and then over sentences. Evaluation is performed on the Kaggle ASAP dataset, training separate models on individual topics, and also reporting some cross-topic results.<\/p>\n

\"\"<\/a><\/p>\n

 <\/p>\n

22.\u00a0Learning Deep Structure-Preserving Image-Text Embeddings<\/strong>
\nLiwei Wang,\u00a0Yin Li,\u00a0Svetlana Lazebnik.\u00a0University of Illinois, Georgia Tech. CVPR 2016.
\nhttp:\/\/www.cv-foundation.org\/…CVPR_2016_paper.pdf<\/a><\/p>\n

The authors present a neural model that maps images and sentences into the same space, in order to perform cross-modal retrieval – find images based on a sentence or find sentences based on an image.<\/p>\n

\"\"<\/a><\/p>\n

The image vectors come from a pre-trained VGG image detection network. The sentence vectors are constructed using Fisher vectors, but they also explore simpler options, such as mean word2vec vectors and tfidf. Both are then mapped through nonlinearities and normalised, and Euclidean distance is used to measure vector similarity. They also investigate the task of mapping noun phrases from the image caption to specific areas of the image.<\/p>\n

23.\u00a0Understanding deep learning requires rethinking generalization<\/strong>
\nChiyuan Zhang,\u00a0Samy Bengio,\u00a0Moritz Hardt,\u00a0Benjamin Recht,\u00a0Oriol Vinyals. Google Brain, DeepMind. ICLR 2017.
\nhttps:\/\/arxiv.org\/pdf\/1611.03530.pdf<\/a><\/p>\n

The authors investigate the generalisation properties of several well-known image recognition networks.<\/p>\n

\"1611.03530\"<\/a><\/p>\n

They show that these networks are able to overfit to the training set with 100% accuracy even if the labels on the images are random, or if the pixels are randomly generated. Regularisation, such as weight decay and dropout, doesn’t stop overfitting as much as expected, still resulting in ~90% accuracy on random training data. They then argue that these models likely make use of massive memorization, in combination with learning low-complexity patterns, in order to perform well on these tasks.<\/p>\n

24.\u00a0Reinforcement Learning with Unsupervised Auxiliary Tasks<\/strong>
\nMax Jaderberg,\u00a0Volodymyr Mnih, Wojciech Marian Czarnecki Tom Schaul, Joel Z Leibo, David Silver & Koray Kavukcuoglu. DeepMind. ICLR 2017.
\nhttps:\/\/arxiv.org\/abs\/1611.05397<\/a><\/p>\n

They describe a version of reinforcement learning where the system also learns to solve some auxiliary tasks, which helps with the main objective.<\/p>\n

\"1611.05397\"<\/a><\/p>\n

In addition to normal Q-learning, which predicts the downstream reward, they have the system learning 1) a separate policy for maximally changing the pixels on the screen, 2) maximally activating units in a hidden layer, and 3) predicting the reward at the next step, using biased sampling. They show that this improves learning speed and performance on Atari games and Labyrinth (a Quake-like 3D game).<\/p>\n

25.\u00a0Modelling metaphor with attribute-based semantics<\/strong>
\nLuana Bulat,\u00a0Stephen Clark,\u00a0Ekaterina Shutova. Cambridge. EACL 2017.
\nhttp:\/\/aclweb.org\/anthology\/E\/E17\/E17-2084.pdf<\/a><\/p>\n

They propose using attribute-based vectors for detecting metaphorical word pairs.<\/p>\n

\"eacl2017\"<\/a>
\nTraditional embeddings (word2vec and count-based) are mapped to attribute vectors, using a supervised system trained on McRae norms. These vectors for a word pair are then given as input to an SVM classifier and trained to detect metaphorical (black humour) vs literal (black dress) word pairs. They show that using the attribute vectors gives higher F score over using the original vector space.<\/p>\n

26.\u00a0Enriching Word Vectors with Subword Information<\/strong>
\nPiotr Bojanowski,\u00a0Edouard Grave,\u00a0Armand Joulin,\u00a0Tomas Mikolov. Facebook. ArXiv 2016.
\nhttps:\/\/arxiv.org\/abs\/1607.04606<\/a><\/p>\n

They extend skip-grams for word embeddings to use character n-grams. Each word is represented as a bag of character n-grams, 3-6 characters long, plus the word itself. Each of these has their own embedding which gets optimised to predict the surrounding context words using skip-gram optimisation. They evaluate on word similarity and analogy tasks, in different languages, and show improvement on most benchmarks.<\/p>\n

27.\u00a0Learning to Compose Words into Sentences with Reinforcement Learning<\/strong>
\nDani Yogatama,\u00a0Phil Blunsom,\u00a0Chris Dyer,\u00a0Edward Grefenstette,\u00a0Wang Ling. DeepMind. ICLR 2017.
\nhttps:\/\/arxiv.org\/abs\/1611.09100<\/a><\/p>\n

The aim is to have the system discover a method for parsing that would benefit a downstream task.<\/p>\n

\"1611.09100\"<\/a><\/p>\n

They construct a neural shift-reduce parser – as it’s moving through the sentence, it can either shift the word to the stack or reduce two words on top of the stack by combining them. A Tree-LSTM is used for composing the nodes recursively. The whole system\u00a0is trained using reinforcement learning, based on an objective function of the downstream task. The model learns parse rules that are beneficial for that specific task, either without any prior knowledge of parsing or by initially training it to act as a regular parser.<\/p>\n

28. Identifying beneficial task relations for multi-task learning in deep neural networks<\/strong>
\nJoachim Bingel, Anders S\u00f8gaard. Copenhagen. EACL 2017.
\nhttp:\/\/www.aclweb.org\/anthology\/E17-2026<\/a><\/p>\n

The authors investigate the benefit of different task combinations when performing multi-task learning.<\/p>\n

\"multitask_tagging\"<\/a><\/p>\n

They experiment with all possible pairs of 10 sequence labeling datasets, switching between the datasets during training. They find that multi-task learning helps more when the main task quickly plateaus while the auxiliary task does not, likely helping the model out of local minima.
\nThere does not seem to be any auxiliary task that would help on all main tasks, but chunking and semantic tagging seem to perform best.<\/p>\n

29. Literal and Metaphorical Senses in Compositional Distributional Semantic Models<\/strong>
\nE. Dar\u00edo Guti\u00e9rrez, Ekaterina Shutova, Tyler Marghetis, Benjamin K. Bergen. UCSD, Cambridge, Bloomington. ACL 2016.
\nhttp:\/\/www.aclweb.org\/anthology\/P16-1018<\/a><\/p>\n

The paper investigates compositional semantic models specialised for metaphors.<\/p>\n

\"metaphor_composition\"<\/a><\/p>\n

They construct a dataset of 8592 adjective-noun phrases, covering 23 different adjectives, annotated for being metaphorical or literal. They then train compositional models to predict the phrase vector based on the noun vector, as a linear combination with an adjective-specific weight matrix. They show that it’s better to learn separate adjective matrices for literal and metaphorical uses of each adjective, even though the amount of training data is smaller.<\/p>\n

30. Data Noising as Smoothing in Neural Network Language Models<\/strong>
\nZiang Xie, Sida I. Wang, Jiwei Li, Daniel Levy, Aiming Nie, Daniel Jurafsky, Andrew Y. Ng. Stanford. ICLR 2017.
\nhttps:\/\/arxiv.org\/abs\/1703.02573<\/a><\/p>\n

The paper investigates better noising techniques for RNN language models.<\/p>\n

\"lm_noising\"<\/a><\/p>\n

A noising technique from previous work would be to randomly replace words in the context or replace them with a blank token. Here they investigate ways of choosing better which words to replace and choosing the replacements from a better distribution, inspired by methods in n-gram smoothing. They show improvement on language modeling (PTB and text8) and machine translation (English-German).<\/p>\n

31. Neural Belief Tracker: Data-Driven Dialogue State Tracking<\/strong>
\nNikola Mrk\u0161i\u0107, Diarmuid \u00d3 S\u00e9aghdha, Tsung-Hsien Wen, Blaise Thomson, Steve Young. Cambridge, Apple. ACL 2017.
\nhttp:\/\/aclweb.org\/anthology\/P\/P17\/P17-1163.pdf<\/a><\/p>\n

They propose neural models for dialogue state tracking, making a binary decision for each possible slot-value pair, based on the latest context from the user and the system. The context utterances and the slot-value option are encoded into vectors, either by summing word representations or using a convnet. These vectors are then further combined to produce a binary output. The systems are evaluated on two dialogue datasets and show improvement over baselines that use hand-constructed lexicons.<\/p>\n

\"dialogue_state_tracking\"<\/a><\/p>\n

32. Neural Architectures for Fine-grained Entity Type Classification<\/strong>
\nSonse Shimaoka, Pontus Stenetorp, Kentaro Inui, Sebastian Riedel. Tohoku, UCL. EACL 2017.
\nhttps:\/\/www.aclweb.org\/anthology\/E17-1119<\/a><\/p>\n

They propose a neural architecture for assigning fine-grained labels to detected entity types. The model combines bidirectional LSTMs, attention over the context sequence, hand-engineered features, and the label hierarchy. They evaluate on Figer and OntoNotes datasets, showing improvements from each of the extensions.<\/p>\n

\"neural_arch_for_ne\"<\/a><\/p>\n

33. Recurrent Additive Networks<\/strong>
\nKenton Lee, Omer Levy, Luke Zettlemoyer. Washington, Allen Institute. ArXiv 2017.
\nhttps:\/\/arxiv.org\/abs\/1705.07393<\/a><\/p>\n

The authors propose a simplified version of LSTMs. Some non-linearities and weighted components are removed, in order to arrive at the recurrent additive network (RAN). The model is evaluated on 3 language modeling datasets: PTB, the billion word benchmark, and character-level Text8.<\/p>\n

34. A Sensitivity Analysis of (and Practitioners’ Guide to) Convolutional Neural Networks for Sentence Classification<\/strong>
\nYe Zhang, Byron Wallace. UT Austin. IJCNLP 2017.
\nhttps:\/\/www.aclweb.org\/anthology\/I\/I17\/I17-1026.pdf<\/a><\/p>\n

The authors perform a hyperparameter search for a single-layer CNN on 9 different sentence classification datasets.
\nThey find that the optimal embedding initialisation, filter size and number of feature maps depends on the dataset and should be chosen through a search; ReLU and tanh are the best activation functions; 1-max pooling is the pooling method; dropout may help when the number of feature maps gets large.<\/p>\n

\"sensitivity_analysis\"<\/a><\/p>\n

35. On Using Monolingual Corpora in Neural Machine Translation<\/strong>
\nCaglar Gulcehre, Orhan Firat, Kelvin Xu, Kyunghyun Cho, Loic Barrault, Huei-Chi Lin, Fethi Bougares, Holger Schwenk, Yoshua Bengio. Montreal, METech, Maine. Computer Speech and Language 2016.
\nhttps:\/\/arxiv.org\/abs\/1503.03535<\/a><\/p>\n

The authors extend a seq2seq model for MT with a language model. They first pre-train a seq2seq model and a neural language model, then train a separate feedforward component that takes the hidden states from both and combines them together to make a prediction. They compare to simply combining the output probabilities from both models (shallow fusion) and show improvement on different MT datasets.<\/p>\n

\"monolingual_corpora_for_mt\"<\/a><\/p>\n

36. Semi-supervised sequence tagging with bidirectional language models<\/strong>
\nMatthew E. Peters, Waleed Ammar, Chandra Bhagavatula, Russell Power. Allen Institute. ACL 2017.
\nhttp:\/\/aclweb.org\/anthology\/P\/P17\/P17-1161.pdf<\/a><\/p>\n

The paper proposes integrating a pre-trained language model into a sequence labeling model. The baseline model for sequence labeling is a two-layer LSTM\/GRU. They concatenate the hidden states from pre-trained language models onto the output of the first LSTM layer. This provides an improvement on NER and chunking tasks.<\/p>\n

\"Semi-supervised<\/a><\/p>\n

37.\u00a0Weakly Supervised Part-of-speech Tagging Using Eye-tracking Data
\n<\/strong>Maria Barrett,\u00a0Joachim\u00a0Bingel,\u00a0Frank Keller,\u00a0Anders S\u00f8gaard. Copenhagen. ACL 2016.
\nhttps:\/\/www.aclweb.org\/anthology\/P\/P16\/P16-2094.pdf<\/a><\/p>\n

The paper explores\u00a0the usefulness of eye tracking for the task of POS tagging.\u00a0The assumption is that readers skip quickly over closed class words, and fixate longer on rare on ambiguous words.<\/p>\n

The experiments are performed on unsupervised POS tagging – a second-order HMM uses constraints on possible tags for each word (based on a dictionary), but no explicit annotated data is required. They show that including the eye tracking features improves performance by quite a bit. Surprisingly, it seems to be better to average eye tracking features over all training tokens of the same type, as opposed to using using the data for each individual token, which means eye tracking is only used during the training stage.<\/p>\n

38. Massive Exploration of Neural Machine Translation Architectures<\/strong>
\nDenny Britz, Anna Goldie, Minh-Thang Luong, Quoc Le. Google Brain. EMNLP 2017.
\nhttp:\/\/aclweb.org\/anthology\/D17-1151<\/a><\/p>\n

Investigates different parameter choices for encoder-decoder NMT models. They find that LSTM is better than GRU, 2 bidirectional layers is enough, additive attention is the best, and a well-tuned beam search is important. They achieve good results on the WMT15 English->German task and release the code.<\/p>\n

\"Massive<\/a><\/p>\n

39. Learning to Reason: End-to-End Module Networks for Visual Question Answering<\/strong>
\nRonghang Hu, Jacob Andreas, Marcus Rohrbach, Trevor Darrell, Kate Saenko. Berkeley, Facebook, Boston. ICCV 2017.
\nhttps:\/\/arxiv.org\/pdf\/1704.05526.pdf<\/a><\/p>\n

A modular neural architecture for visual question answering. A seq2seq component predicts the sequence of neural modules (eg find() and compare()) based on the textual question, which are then dynamically combined and trained end-to-end. Achieves good results on three separate benchmarks that focus on reasoning about the image.<\/p>\n

\"Learning<\/a><\/p>\n

40. Automatic Annotation and Evaluation of Error Types for Grammatical Error Correction<\/strong>
\nChristopher Bryant, Mariano Felice, Ted Briscoe. Cambridge. ACL 2017.
\nhttp:\/\/aclweb.org\/anthology\/P\/P17\/P17-1074.pdf<\/a><\/p>\n

A toolkit for automatically annotating error correction data with error types. It takes original and corrected sentences as input, aligns them to infer error spans, and uses rules to assign error types. They use the tool to perform fine-grained evaluation of CoNLL-14 shared task participants.<\/p>\n

41. Dynamic Evaluation of Neural Sequence Models<\/strong>
\nBen Krause, Emmanuel Kahembwe, Iain Murray, Steve Renals. Edinburgh. ArXiv 2017.
\nhttps:\/\/arxiv.org\/abs\/1709.07432<\/a><\/p>\n

Updating the parameters in a LSTM language model based on the observed sequence during testing. A slice of text is first processed and then used for a gradient descent update step. A regularisation term is also proposed which draws the parameters back towards the original model.<\/p>\n

\"Dynamic<\/a><\/p>\n

42. Unsupervised Machine Translation Using Monolingual Corpora Only<\/strong>
\nGuillaume Lample, Ludovic Denoyer, Marc’Aurelio Ranzato. Facebook, Sorbonne. ArXiv 2017.
\nhttps:\/\/arxiv.org\/abs\/1711.00043<\/a><\/p>\n

The model learns to translate using a seq2seq model, an autoencoder objective, and an adversarial objective for language identification.
\nThe system is trained to correct noisy versions of its own output and iteratively improves performance.
\nDoes not require parallel corpora, but relies on a separate method for inducing a parallel dictionary that bootstraps the translation.<\/p>\n

\"Unsupervised<\/a><\/p>\n

43. Assessing the Ability of LSTMs to Learn Syntax-Sensitive Dependencies<\/strong>
\nTal Linzen, Emmanuel Dupoux, Yoav Goldberg. ENS, Bar Ilan. TACL 2017.
\nhttp:\/\/aclweb.org\/anthology\/Q\/Q16\/Q16-1037.pdf<\/a><\/p>\n

Investigation of how well LSTMs capture long-distance dependencies. The task is to predict verb agreement (singular or plural) when the subject noun is separated by different numbers of distractors. They find that an LSTM trained explicitly for this task manages to handle even most of the difficult cases, but a regular language model is more prone to being misled by the distractors.<\/p>\n

\"Assessing<\/a><\/p>\n

44. Multiple Instance Learning Networks for Fine-Grained Sentiment Analysis<\/strong>
\nStefanos Angelidis, Mirella Lapata. Edinburgh. ArXiv 2017.
\nhttps:\/\/arxiv.org\/pdf\/1711.09645.pdf<\/a><\/p>\n

A model for document sentiment classification which can also return sentence-level sentiment predictions. They construct sentence-level representations using a convnet, use this to predict a sentence-level probability distribution over possible sentiment labels, and then combine these over all sentences either with a fixed weight vector or using an attention mechanism. They release a new dataset of 200 documents annotated on the level of sentences and discourse units.<\/p>\n

\"Multiple<\/a><\/p>\n

45. Learning how to Active Learn: A Deep Reinforcement Learning Approach<\/strong>
\nMeng Fang, Yuan Li, Trevor Cohn. Melbourne. EMNLP 2017.
\nhttp:\/\/aclweb.org\/anthology\/D\/D17\/D17-1063.pdf<\/a><\/p>\n

Active learning (choosing which examples to annotate for training) is proposed as a reinforcement learning problem. The Q-learning network predicts for each sentence whether it should be annotated, and is trained based on the performance improvement from the main task. Evaluation is done on NER, with experiments on transferring the trained Q-learning function to other languages.<\/p>\n

\"Learning<\/a><\/p>\n

46. On the State of the Art of Evaluation in Neural Language Models<\/strong>
\nG\u00e1bor Melis, Chris Dyer, Phil Blunsom. Deepmind, Oxford. ArXiv 2017.
\nhttps:\/\/arxiv.org\/pdf\/1707.05589.pdf<\/a><\/p>\n

Comparison of three recurrent architectures for language modelling: LSTMs, Recurrent Highway Networks and the NAS architecture. Each model goes through a substantial hyperparameter search, under the constraint that the total number of parameters is kept constant. They conclude that basic LSTMs still outperform other architectures and achieve state-of-the-art perplexities on two datasets.<\/p>\n

47. Dynamic Routing Between Capsules<\/strong>
\nSara Sabour, Nicholas Frosst, Geoffrey E Hinton. Google Brain. NIPS 2017.
\nhttps:\/\/arxiv.org\/pdf\/1710.09829.pdf<\/a><\/p>\n

An attention-based architecture for combining information from different convolutional layers. The attention values are calculated using an iterative process, making use of a custom squashing function. The evaluations on MNIST show robustness to affine transformations.<\/p>\n

48. Multilingual Part-of-Speech Tagging with Bidirectional Long Short-Term Memory Models and Auxiliary Loss<\/strong>
\nBarbara Plank,\u00a0Anders S\u00f8gaard,\u00a0Yoav Goldberg. Groningen, Copenhagen, Bar-Ilan. ACL 2016.
\nhttp:\/\/aclweb.org\/anthology\/P\/P16\/P16-2067.pdf<\/a><\/p>\n

Doing\u00a0POS tagging using a bidirectional LSTM with word- and character-based embeddings. They add an extra component to the loss function – predicting a frequency class for each word, together with their POS tag. Results show that overall performance remains\u00a0similar, but there’s an improvement in tagging accuracy for low-frequency words.<\/p>\n

\"sequence_tagging\"<\/a><\/p>\n

49. Emergent Translation in Multi-Agent Communication<\/strong>
\nJason Lee, Kyunghyun Cho, Jason Weston, Douwe Kiela. Facebook. ArXiv 2017.
\nhttps:\/\/arxiv.org\/pdf\/1710.06922.pdf<\/a><\/p>\n

Learning to translate using two monolingual image captioning datasets and pivoting through images. The model encodes an image and generates a caption in language A, this is then encoded into the same space as language B and the representation is optimised to be similar to the correct image. The model is trained end-to-end using Gumbel-softmax.<\/p>\n

\"Emergent<\/a><\/p>\n

50. Efficient softmax approximation for GPUs<\/strong>
\nEdouard Grave, Armand Joulin, Moustapha Ciss\u00e9, David Grangier, Herv\u00e9 J\u00e9gou. Facebook. ICML 2017.
\nhttps:\/\/arxiv.org\/pdf\/1609.04309.pdf<\/a><\/p>\n

Modification of the 2-level hierarchical softmax for better efficiency. An equation of computational complexity is used to find the optimal number of words in each class. In addition, the most common words are considered on the same level as other classes.<\/p>\n

\"Efficient<\/a><\/p>\n

51. Semi-supervised Multitask Learning for Sequence Labeling<\/strong>
\nMarek Rei. Cambridge. ACL 2017.
\nhttp:\/\/aclweb.org\/anthology\/P\/P17\/P17-1194.pdf<\/a><\/p>\n

Incorporating an unsupervised language modeling objective to help train a bidirectional LSTM for sequence labeling. At the same time as training the tagger, the forward-facing LSTM is optimised to predict the next word and the backward-facing LSTM is optimised to predict the previous word. The model learns a better composition function and improves performance on NER, error detection, chunking and POS-tagging, without using additional data.<\/p>\n

\"Semi-supervised<\/a><\/p>\n

52. Grasping the Finer Point: A Supervised Similarity Network for Metaphor Detection<\/strong>
\nMarek Rei, Luana Bulat, Douwe Kiela, Ekaterina Shutova. Cambridge, Facebook. EMNLP 2017.
\nhttp:\/\/aclweb.org\/anthology\/D\/D17\/D17-1162.pdf<\/a><\/p>\n

A specialised architecture for detecting metaphorical phrases. Uses a gating mechanism to condition one word based on the other, a neural version of weighted cosine similarity to make a prediction and hinge loss to optimise the model. Achieves high results on detecting metaphorical adjective-noun, verb-object and verb-subject phrases.<\/p>\n

\"Grasping<\/a><\/p>\n

53. Neural Sequence-Labelling Models for Grammatical Error Correction<\/strong>
\nHelen Yannakoudakis, Marek Rei, \u00d8istein E. Andersen, Zheng Yuan. Cambridge. EMNLP 2017.
\nhttp:\/\/aclweb.org\/anthology\/D\/D17\/D17-1297.pdf<\/a><\/p>\n

Using error detection to improve error correction. A neural sequence labeling model is used to find correctness probabilities for every token, which are then used to rerank possible correction candidates. The process consistently improves the performance of different correction systems.<\/p>\n

\"Neural<\/a><\/p>\n

54. Artificial Error Generation with Machine Translation and Syntactic Patterns<\/strong>
\nMarek Rei, Mariano Felice, Zheng Yuan, Ted Briscoe. Cambridge. BEA 2017.
\nhttp:\/\/aclweb.org\/anthology\/W\/W17\/W17-5032.pdf<\/a><\/p>\n

Investigating methods for generating artificial data in order to train better systems for detecting grammatical errors. The first approach uses regular machine translation, essentially translating from correct English to incorrect English. The second method uses local patterns with slots and POS tags to insert errors into new text.<\/p>\n

\"Artificial<\/a><\/p>\n

55. Auxiliary Objectives for Neural Error Detection Models<\/strong>
\nMarek Rei, Helen Yannakoudakis. Cambridge. BEA 2017.
\nhttp:\/\/aclweb.org\/anthology\/W\/W17\/W17-5004.pdf<\/a><\/p>\n

Investigating a range of auxiliary objectives for training a sequence labeling system for error detection. Automatically generated dependency relations and POS tags perform surprisingly well as gold labels for multi-task learning. Learning different objectives at the same time works better than doing them in sequence or switching.<\/p>\n

\"Auxiliary<\/a><\/p>\n

56. An Error-Oriented Approach to Word Embedding Pre-Training<\/strong>
\nYoumna Farag, Marek Rei, Ted Briscoe. Cambridge. BEA 2017.
\nhttp:\/\/aclweb.org\/anthology\/W\/W17\/W17-5016.pdf<\/a><\/p>\n

Introduces a process for pre-training word embeddings with an objective that optimises them to distinguish between grammatical and ungrammatical sequences. This is then extended to also distinguish between correct and incorrect versions of the same sentence. The embeddings are then used in a network for essay scoring, improving performance compared to previous methods.<\/p>\n

\"An<\/a><\/p>\n

57. Detecting Off-topic Responses to Visual Prompts<\/strong>
\nMarek Rei. Cambridge. BEA 2017.
\nhttp:\/\/aclweb.org\/anthology\/W\/W17\/W17-5020.pdf<\/a><\/p>\n

A neural architecture for detecting off-topic written responses, with respect to visual prompts. The text is composed with an LSTM and then used to condition the image representation. The two representations are then compared to calculate a confidence score for the text being written in response to the prompt image.<\/p>\n

\"Detecting<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"

Staying on top of recent work is an important part of being a good researcher, but this can be quite difficult. Thousands of new papers…<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[],"class_list":["post-717","post","type-post","status-publish","format-standard","hentry","category-uncategorized"],"yoast_head":"\n57 Summaries of Machine Learning and NLP Research - Marek Rei<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.marekrei.com\/blog\/paper-summaries\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"57 Summaries of Machine Learning and NLP Research - Marek Rei\" \/>\n<meta property=\"og:description\" content=\"Staying on top of recent work is an important part of being a good researcher, but this can be quite difficult. Thousands of new papers…\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.marekrei.com\/blog\/paper-summaries\/\" \/>\n<meta property=\"og:site_name\" content=\"Marek Rei\" \/>\n<meta property=\"article:published_time\" content=\"2018-01-17T15:02:16+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2019-09-27T23:22:28+00:00\" \/>\n<meta property=\"og:image\" content=\"http:\/\/www.marekrei.com\/blog\/wp-content\/uploads\/2018\/01\/cnn_daily_mail-300x151.png\" \/>\n<meta name=\"author\" content=\"Marek\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Marek\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"30 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\/\/www.marekrei.com\/blog\/paper-summaries\/\",\"url\":\"https:\/\/www.marekrei.com\/blog\/paper-summaries\/\",\"name\":\"57 Summaries of Machine Learning and NLP Research - Marek Rei\",\"isPartOf\":{\"@id\":\"https:\/\/www.marekrei.com\/blog\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/www.marekrei.com\/blog\/paper-summaries\/#primaryimage\"},\"image\":{\"@id\":\"https:\/\/www.marekrei.com\/blog\/paper-summaries\/#primaryimage\"},\"thumbnailUrl\":\"http:\/\/www.marekrei.com\/blog\/wp-content\/uploads\/2018\/01\/cnn_daily_mail-300x151.png\",\"datePublished\":\"2018-01-17T15:02:16+00:00\",\"dateModified\":\"2019-09-27T23:22:28+00:00\",\"author\":{\"@id\":\"https:\/\/www.marekrei.com\/blog\/#\/schema\/person\/a145eb0a06ed4acf5b0f84a24b7a1191\"},\"breadcrumb\":{\"@id\":\"https:\/\/www.marekrei.com\/blog\/paper-summaries\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/www.marekrei.com\/blog\/paper-summaries\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.marekrei.com\/blog\/paper-summaries\/#primaryimage\",\"url\":\"http:\/\/www.marekrei.com\/blog\/wp-content\/uploads\/2018\/01\/cnn_daily_mail-300x151.png\",\"contentUrl\":\"http:\/\/www.marekrei.com\/blog\/wp-content\/uploads\/2018\/01\/cnn_daily_mail-300x151.png\"},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/www.marekrei.com\/blog\/paper-summaries\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/www.marekrei.com\/blog\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"57 Summaries of Machine Learning and NLP Research\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/www.marekrei.com\/blog\/#website\",\"url\":\"https:\/\/www.marekrei.com\/blog\/\",\"name\":\"Marek Rei\",\"description\":\"Thoughts on Machine Learning and Natural Language Processing\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/www.marekrei.com\/blog\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Person\",\"@id\":\"https:\/\/www.marekrei.com\/blog\/#\/schema\/person\/a145eb0a06ed4acf5b0f84a24b7a1191\",\"name\":\"Marek\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.marekrei.com\/blog\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/48a65414bfda6485aaa0703e548de0ed25292b5fe0d979ed8c28ad83cf5a82c0?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/48a65414bfda6485aaa0703e548de0ed25292b5fe0d979ed8c28ad83cf5a82c0?s=96&d=mm&r=g\",\"caption\":\"Marek\"},\"url\":\"https:\/\/www.marekrei.com\/blog\/author\/marek\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"57 Summaries of Machine Learning and NLP Research - Marek Rei","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.marekrei.com\/blog\/paper-summaries\/","og_locale":"en_US","og_type":"article","og_title":"57 Summaries of Machine Learning and NLP Research - Marek Rei","og_description":"Staying on top of recent work is an important part of being a good researcher, but this can be quite difficult. Thousands of new papers…","og_url":"https:\/\/www.marekrei.com\/blog\/paper-summaries\/","og_site_name":"Marek Rei","article_published_time":"2018-01-17T15:02:16+00:00","article_modified_time":"2019-09-27T23:22:28+00:00","og_image":[{"url":"http:\/\/www.marekrei.com\/blog\/wp-content\/uploads\/2018\/01\/cnn_daily_mail-300x151.png"}],"author":"Marek","twitter_misc":{"Written by":"Marek","Est. reading time":"30 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/www.marekrei.com\/blog\/paper-summaries\/","url":"https:\/\/www.marekrei.com\/blog\/paper-summaries\/","name":"57 Summaries of Machine Learning and NLP Research - Marek Rei","isPartOf":{"@id":"https:\/\/www.marekrei.com\/blog\/#website"},"primaryImageOfPage":{"@id":"https:\/\/www.marekrei.com\/blog\/paper-summaries\/#primaryimage"},"image":{"@id":"https:\/\/www.marekrei.com\/blog\/paper-summaries\/#primaryimage"},"thumbnailUrl":"http:\/\/www.marekrei.com\/blog\/wp-content\/uploads\/2018\/01\/cnn_daily_mail-300x151.png","datePublished":"2018-01-17T15:02:16+00:00","dateModified":"2019-09-27T23:22:28+00:00","author":{"@id":"https:\/\/www.marekrei.com\/blog\/#\/schema\/person\/a145eb0a06ed4acf5b0f84a24b7a1191"},"breadcrumb":{"@id":"https:\/\/www.marekrei.com\/blog\/paper-summaries\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.marekrei.com\/blog\/paper-summaries\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.marekrei.com\/blog\/paper-summaries\/#primaryimage","url":"http:\/\/www.marekrei.com\/blog\/wp-content\/uploads\/2018\/01\/cnn_daily_mail-300x151.png","contentUrl":"http:\/\/www.marekrei.com\/blog\/wp-content\/uploads\/2018\/01\/cnn_daily_mail-300x151.png"},{"@type":"BreadcrumbList","@id":"https:\/\/www.marekrei.com\/blog\/paper-summaries\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/www.marekrei.com\/blog\/"},{"@type":"ListItem","position":2,"name":"57 Summaries of Machine Learning and NLP Research"}]},{"@type":"WebSite","@id":"https:\/\/www.marekrei.com\/blog\/#website","url":"https:\/\/www.marekrei.com\/blog\/","name":"Marek Rei","description":"Thoughts on Machine Learning and Natural Language Processing","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.marekrei.com\/blog\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Person","@id":"https:\/\/www.marekrei.com\/blog\/#\/schema\/person\/a145eb0a06ed4acf5b0f84a24b7a1191","name":"Marek","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.marekrei.com\/blog\/#\/schema\/person\/image\/","url":"https:\/\/secure.gravatar.com\/avatar\/48a65414bfda6485aaa0703e548de0ed25292b5fe0d979ed8c28ad83cf5a82c0?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/48a65414bfda6485aaa0703e548de0ed25292b5fe0d979ed8c28ad83cf5a82c0?s=96&d=mm&r=g","caption":"Marek"},"url":"https:\/\/www.marekrei.com\/blog\/author\/marek\/"}]}},"_links":{"self":[{"href":"https:\/\/www.marekrei.com\/blog\/wp-json\/wp\/v2\/posts\/717","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.marekrei.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.marekrei.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.marekrei.com\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.marekrei.com\/blog\/wp-json\/wp\/v2\/comments?post=717"}],"version-history":[{"count":94,"href":"https:\/\/www.marekrei.com\/blog\/wp-json\/wp\/v2\/posts\/717\/revisions"}],"predecessor-version":[{"id":1293,"href":"https:\/\/www.marekrei.com\/blog\/wp-json\/wp\/v2\/posts\/717\/revisions\/1293"}],"wp:attachment":[{"href":"https:\/\/www.marekrei.com\/blog\/wp-json\/wp\/v2\/media?parent=717"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.marekrei.com\/blog\/wp-json\/wp\/v2\/categories?post=717"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.marekrei.com\/blog\/wp-json\/wp\/v2\/tags?post=717"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}