machine learning - Doc2vec: Only 10 docvecs in gensim doc2vec model?

I used gensim fit a doc2vec model, with tagged document (length>10) as training data. The target is to get doc vectors of all training docs, but only 10 vectors can be found in model.docvecs.The example of training data (length>10)docs = ['This is a sentence', 'This is another sentence', ....]with some pre-treatmentdoc_=[d.strip().split(" ") for d in doc]doc_tagged = []for i in range(len(doc_)): tagd = TaggedDocument(doc_[i],str(i)) doc_tagged.append(tagd)tagged docsTaggedDocument(words=array(['a', 'b', 'c', ..., ], dtype='<U32'), tags='1...Read more

machine learning - TensorFlow error: "logits and labels must be same size", warmspringwinds "tutorial"

I'm currently following this tutorial and after I did some changes because of the tensorflow update, I got this error: tensorflow.python.framework.errors_impl.InvalidArgumentError: logits and labels must be same size: logits_size=[399360,2] labels_size=[409920,2] [[Node: SoftmaxCrossEntropyWithLogits = SoftmaxCrossEntropyWithLogits[T=DT_FLOAT, _device="/job:localhost/replica:0/task:0/cpu:0"](Reshape_2, Reshape_3)]]. Can anyone help me with this one?Changes in the code:#Replaced concat_dim=2 with axis=2combined_mask = tf.concat(axis=2, v...Read more

What are advantages of Artificial Neural Networks over Support Vector Machines?

ANN (Artificial Neural Networks) and SVM (Support Vector Machines) are two popular strategies for supervised machine learning and classification. It's not often clear which method is better for a particular project, and I'm certain the answer is always "it depends." Often, a combination of both along with Bayesian classification is used.These questions on Stackoverflow have already been asked regarding ANN vs SVM:ANN and SVM classificationwhat the difference among ANN, SVM and KNN in my classification questionSupport Vector Machine or Artificia...Read more

machine learning - Clarification on a Neural Net that plays Snake

I'm new to neural networks/machine learning/genetic algorithms, and for my first implementation I am writing a network that learns to play snake (An example in case you haven't played it before) I have a few questions that I don't fully understand:Before my questions I just want to make sure I understand the general idea correctly. There is a population of snakes, each with randomly generated DNA. The DNA is the weights used in the neural network. Each time the snake moves, it uses the neural net to decide where to go (using a bias). When the p...Read more

machine learning - How does Apple find dates, times and addresses in emails?

In the iOS email client, when an email contains a date, time or location, the text becomes a hyperlink and it is possible to create an appointment or look at a map simply by tapping the link. It not only works for emails in English, but in other languages also. I love this feature and would like to understand how they do it. The naive way to do this would be to have many regular expressions and run them all. However I this is not going to scale very well and will work for only a specific language or date format, etc. I think that Apple must be...Read more

machine learning - Concatenating two doc2vec models: Vector dimensions doubled

I have a question regarding concatenating two doc2vec models. I followed the official gensim IMDB example on doc2vec and implemented example data.When concatenating two models (PV-DM + PV-DBOW), as outlined in the original paper, I wondered that the concatenated model appears not to have 200-dim, like the two input models, but 400-dim:Shape Train(11948, **400**)Shape Test(2987, **400**)The input shapes were each:np.asarray(X_train).shape)(11948, **200**)(2987, **200**)Is this correct? I expected the number of dimensions to be 200 again....Read more

machine learning - Is Word2Vec and Glove vectors are suited for Entity Recognition?

I am working on Named Entity Recognition. I evaluated libraries, such as MITIE, Stanford NER , NLTK NER etc., which are built upon conventional nlp techniques. I also looked at deep learning models such as word2vec and Glove vectors for representing words in vector space, they are interesting since they provide the information about the context of a word, but specifically for the task of NER, I think its not well suited. Since all these vector models create a vocab and corresponding vector representation. If any word failed to be in the vocabul...Read more

machine learning - NLP - Embeddings selection of `start` and `end` of sentence tokens

Suppose we're training a neural network model to learn the mapping from the following input to output, where the output is Name Entity (NE).Input: EU rejects German call to boycott British lamb .Output: ORG O MISC O O O MISC O OA sliding window is created to capture the context information and its outcomes are fed into the training model as model_input. The sliding window generates results as following: [['<s>', '<s>', 'EU', 'rejects', 'German'],\ ['<s>', 'EU', 'rejects', 'German', 'call'],\ ['EU', 'rejects', 'German', 'call',...Read more

machine learning - Using pre-trained word2vec with LSTM for word generation

LSTM/RNN can be used for text generation.This shows way to use pre-trained GloVe word embeddings for Keras model.How to use pre-trained Word2Vec word embeddings with Keras LSTMmodel? This post did help.How to predict / generate next word when the model is provided with the sequence of words as its input?Sample approach tried:# Sample code to prepare word2vec word embeddings import gensimdocuments = ["Human machine interface for lab abc computer applications", "A survey of user opinion of computer system response time", ...Read more

machine learning - Character-Word Embeddings from lm_1b in Keras

I would like to use some pre-trained word embeddings in a Keras NN model, which have been published by Google in a very well known article. They have provided the code to train a new model, as well as the embeddings here.However, it is not clear from the documentation how to retrieve an embedding vector from a given string of characters (word) from a simple python function call. Much of the documentation seems to center on dumping vectors to a file for an entire sentence presumably for sentimental analysis. So far, I have seen that you can ...Read more

machine learning - What is the best way to handle missing words when using word embeddings?

I have a set of pre-trained word2vec word vectors and a corpus. I want to use the word vectors to represent words in the corpus. The corpus has some words in it that I don't have trained word vectors for. What's the best way to handle those words for which there is no pre-trained vector?I've heard several suggestions. use a vector of zeros for every missing worduse a vector of random numbers for every missing word (with a bunch of suggestions on how to bound those randoms)an idea I had: take a vector whose values are the mean of all values in t...Read more

machine learning - Load vectors into gensim Word2Vec model - not KeyedVectors

I'm attempting to load some pre-trained vectors into a gensim Word2Vec model, so they can be retrained with new data. My understanding is I can do the retraining with gensim.Word2Vec.train(). However, the only way I can find to load the vectors is with gensim.models.KeyedVectors.load_word2vec_format('path/to/file.bin', binary=True) which creates an object of what is usually the wv attribute of a gensim.Word2Vec model. But this object, on it's own, does not have a train() method, which is what I need to retrain the vectors. So how do I get these...Read more