I recently gave this tutorial as a talk\u00a0in University of Cambridge and it turned out to be way more popular than expected. In order to give more people access to the material, I’m now writing it up as a blog post.<\/p>\n
I do not claim to know everything about Theano, and I constantly learn new things myself. If you find any errors or have suggestions on how to improve this tutorial, do let me know.<\/p>\n
The code examples can be found in the Github repository:\u00a0https:\/\/github.com\/marekrei\/theano-tutorial<\/a><\/p>\n Theano is a Python library for efficiently handling mathematical expressions involving multi-dimensional arrays (also known as tensors). It is a common choice\u00a0for implementing neural network models. Theano has been developed in University of Montreal, in a group led by Yoshua Bengio, since 2008.<\/span><\/p>\n Some of the features include:<\/p>\n Theano is a Python library, so let’s go over some important points in Python.<\/p>\n In order to use Theano, you will need to install the dependencies and install Theano itself. If you’re using Ubuntu (tested for 14.04), you might get away with just running these two commands:<\/p>\n If that doesn’t work for you, take a look at the original Theano homepage, which contains instructions for various platforms: To use Theano in your Python script, include it using:<\/p>\n Here is the smallest example I could come up with, which uses Theano and actually does something:<\/p>\n So what’s happening here?<\/p>\n We first define\u00a0a Theano variable x to be a vector of 32-bit floats, and give it name ‘x’:<\/p>\n Next, we create a Theano variable W, assign its value to be vector [0.2, 0.7], and name it ‘W’:<\/p>\n We define y to be the sum of all elements in the element-wise multiplication of x and W:<\/p>\n We define a Theano function f, which takes as input x and outputs y:<\/p>\n Then\u00a0call this function, giving as the argument vector\u00a0[1.0, 1.0], essentially setting the value of variable x:<\/p>\n The script prints out the summed product of [0.2, 0.7] and\u00a0[1.0, 1.0], which is:<\/p>\n Don’t worry if the code\u00a0doesn’t fully make sense. We’ll go over the important parts in more detail.<\/p>\n I’d say this section contains\u00a0the most crucial part to\u00a0understanding Theano.<\/p>\n When we are creating a model\u00a0with Theano, we first define a symbolic graph of all variables and operations that need to be performed. And then we can apply this graph on specific inputs to get outputs.<\/p>\n For example, what do you think happens when this line of Theano code is executed in our script?<\/p>\n The system takes x and W, multiplies them together and sums the values. Right?<\/p>\n NOPE<\/strong><\/p>\n Instead, we create a Theano object y that knows its values can be calculated as the dot-product of x and W. But the required mathematical operations are not performed here. In fact, when this line was executed in our example code above, x didn’t even have a value yet.<\/p>\n By chaining up various operations, we are creating a graph of all the variables and functions that need to be used to reach the output values. This symbolic graph is also the reason why we can only use Theano-specific operations when defining our models. If we tried to integrate functions from some random Python library into our network,\u00a0they would attempt to perform the calculations immediately, instead of returning a Theano variable as needed. Exceptions do exist\u00a0– Theano overrides some basic Python operators to act as expected, and NumPy<\/a> is quite well integrated with Theano.<\/p>\n We can define variables which don\u2019t have any values yet. Normally, these would be used for inputs to our network.<\/p>\n The variables have to be of a specific type though. For example, here we define variable x to be a vector of 32-bit floats, and give it name ‘x’:<\/p>\n The names are generally useful for debugging and informative error messages. Theano won’t have access to your Python variable names, so you have to assign explicit Theano names for each variable if you want them to be referred to as something more useful\u00a0than just\u00a0“a tensor”.<\/p>\n There are a number of different variable types available, just have a look at the list here<\/a>. Some of the more popular ones include:<\/p>\n [table width=”500″ colwidth=”100|50|50″ colalign=”left|center|center”] You can also define a generic vector (or tensor) and set the type with an argument:<\/p>\n If you don’t set the dtype<\/em>, you will create vectors of type config.floatX. This will become relevant in\u00a0the section about GPUs.<\/p>\n We can also define shared variables, which are shared between different functions and different function calls. Normally, these would be used for weights in our neural network. Theano will automatically try to move shared variables to the GPU, provided one is available, in order to speed up computation.<\/p>\n Here we define a shared variable and set its value to [0.2, 0.7].<\/p>\n The values in shared variables can be accessed and modified outside of our Theano functions using these commands:<\/p>\n Theano functions are basically hooks for interacting with the symbolic graph. Commonly, we use them for passing input into our network and collecting the resulting output.<\/p>\n Here we define a Theano function f that takes x as input and returns y as output:<\/p>\n The first parameter is the list of input variables, and the second parameter is the list of output variables. Although if there’s only one output variable (like now) we don’t need to make it into a list.<\/p>\n When we construct a function, Theano takes over and performs some of its own magic. It builds the computational graph and optimises it as much as possible. It restructures\u00a0mathematical operations to make them faster and more stable, compiles some parts to C, moves some tensors to the GPU, etc.<\/p>\n Theano compilation can be controlled by setting the value of mode<\/em> in the environement variable THEANO_FLAGS:<\/p>\n Here’s a minimal script for actually training something in Theano. We will be training the weights in W using gradient descent, so that the result from the model would be 20<\/em> instead of the original 0.9<\/em>.<\/p>\n We create a second input variable called target<\/em>, which will act as the target value we use for training:<\/p>\n In order to train the model, we need a cost function. Here we use a simple squared distance from the target:<\/p>\n Next, we want to calculate the partial gradients for the parameters that will be updated, with respect to the cost function. Luckily, Theano will do that for us. We simply call the grad<\/em> function, pass in the real-valued cost and a list of all the variables we want gradients for, and it will return a list of those gradients:<\/p>\n Now let’s define a symbolic variable for what the updated version of the parameters will look like. Using gradient descent, the update rule is to subtract the gradient, multiplied by the learning rate:<\/p>\n And next\u00a0we create a list of updates. More specifically, a list of tuples where the first element is the variable we want to update, and the second element is a variable containing the values that we want the first variable to contain after the update. This is just a syntax that Theano requires.<\/p>\n Have to define a Theano function again, with a couple of changes:<\/p>\n It now takes two input arguments – one for the input vector, and another for the target value used for training. And the list of updates also gets attached to the function as well. Every time this function is called, we pass in values for x<\/em> and target<\/em>, get back the value for y<\/em> as output, and Theano performs all the updates in the update list.<\/p>\n In order to train the parameters, we repeatedly call this function (10 times in this case). Normally, we’d pass in different examples from our training data, but for this example we use the same x=[1.0, 1.0] and target=20 each time:<\/p>\n When the script is executed, the output looks like this:<\/p>\n The first time the function is called, the output value is still 0.9 (like in the previous example), because the updates have not been applied yet. But with each consecutive step, the output value becomes closer and closer to the desired target 20.<\/p>\n This covers the basic logic behind building models with Theano. The example was very simple, but we are free to define increasingly complicated networks, as long as we use Theano-specific functions. Now let’s look at some of these building blocks that we have available.<\/p>\n Evaluate the value of a Theano variable<\/strong><\/p>\n The eval() function forces the Theano variable to calculate and return its actual (numerical) value. If we try to just print the variable a<\/em>, we only print its name. But if we use eval(), we get the actual square matrix that it is initialised to.<\/p>\n This eval() function isn’t really used for building models, but it can be useful for debugging and learning how Theano works. In the examples below, I will be using the matrix a<\/em> and the eval() function to print the value of each variable and demonstrate how different operations work.<\/p>\n Basic element-wise operations: + – * \/<\/strong><\/p>\n Dot product<\/strong><\/p>\n Activation functions<\/strong><\/p>\n Softmax (row-wise)<\/strong><\/p>\n Sum<\/strong><\/p>\n Max<\/strong><\/p>\n Argmax<\/strong><\/p>\n Reshape<\/strong><\/p>\n We sometimes need to change the dimensions of a tensor and reshape() allows us to do that. It takes as input a tuple containing the new shape and returns a new tensor with that shape. In the first example below, we shape a square matrix into a 1×4 matrix. In the second example, we use -1 which means “as big as the dimension needs to be”.<\/p>\n Zeros-like, ones-like<\/strong><\/p>\n These functions create new tensors with the same shape but all values set to zero or one.<\/p>\n Reorder the tensor dimensions<\/strong><\/p>\n Sometimes we need to reorder the dimensions in a tensor. In the examples below, the dimensions in a two-dimensional matrix are first swapped. Then, ‘x’ is used to create a brand new dimension.<\/p>\n Indexing<\/strong><\/p>\n Using Python indexing tricks can make life so much easier. In the example below, we make a separate list b<\/em>\u00a0containing line numbers, and use it to construct a new matrix which contains exactly the lines we want from the original matrix. This can be useful when dealing with word embeddings – we can put word ids into a list and use this to retrieve exactly the correct sequence of embeddings from the whole embedding matrix.<\/p>\n For assignment, we can’t do this:<\/p>\n But instead, we can use set_subtensor(), which takes as arguments the selection of the original matrix that we want to reassign, and the value we want to assign it to. It returns a new tensor that has the corresponding values modified.<\/p>\n At this point, it’s time to move on to some more realistic examples.<\/p>\n Take a look at the code for a very basic classifier<\/a>, which\u00a0tries to train a small network on a tiny (but real) dataset. I won’t walk you through it line-by-line any more; you’ve learned all the necessary parts by now and there are comments in the code as well.<\/p>\n The task is to predict whether the GDP per capita for a country is more than the average GDP, based on the following features:<\/p>\n The data\/<\/em> directory contains the files for training (121 countries) and testing (40 countries). Each row represents one country, the first column is the label, followed by the features. The feature values have been normalised, by subtracting the mean and dividing by the standard deviation. The label is 1 if the GDP is more than average, and 0 otherwise.<\/p>\n Once you clone the github repository (or just download the data files), you can run the script with:<\/p>\n The script will print information about 10 training epochs and the result on the test set:<\/p>\n One more important operation\u00a0to cover is scan, which can be used to create various recurrent functions: RNN, GRU, LSTM, etc.<\/p>\n Here is sample code for using scan to define a simple RNN over word vectors in the\u00a0input_vectors matrix:<\/p>\n The scan function is called on line 10\u00a0and it takes 4 important arguments:<\/p>\n We’ve defined the helper function rnn_step on line 1, which gets called on each row of\u00a0our input matrix. The scan function will be calling this rnn_step function internally, so we need to accept any arguments in the same order as Theano passes them. This is just something you need to know when dealing with scan. The order is as follows:<\/p>\n What comes out from the scan function contains the hidden states (eg the rnn_step outputs) at each step. Not just the last step, but all of them. So if you only want the last step, you need to explicitly retrieve it by indexing from -1 (the last element). Theano is actually smart enough to figure out that you’re only using the last result, and will optimise to discard all the intermediate ones.<\/p>\n In order to construct the weight matrices, I’m using a helper function ( Time to look at some more code, this time using recurrent functions and scan. The script is available at the Github repository<\/a>. In this example, I’m using Gated Recurrent Units (GRU) from \u201cLearning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation\u201d (Cho et al, 2014<\/a>), which are essentially a simpler versions of LSTMs.<\/p>\n The task is to classify sentences into 5 classes, based on their fine-grained sentiment (very negative, slightly negative, neutral, slightly positive, very positive). We use the dataset published in “Recursive Deep Models for Semantic Compositionality Over a Sentiment Treebank” (Socher et al., 2013<\/a>).<\/p>\n Start by downloading the dataset from http:\/\/nlp.stanford.edu\/sentiment\/<\/a> (the main zip file) and unpack it somewhere. Then, create training and test splits in the format that is more suitable for us, using the provided script in the repository<\/a>:<\/p>\n Now we can run the classifier with:<\/p>\n The script will train for 3 passes over the training data, and will then print performance on the test data.<\/p>\n The accuracy on the test set is about 38%, which isn’t a great result. But it is quite a difficult task – the current state-of-the-art system (Tai ei al., 2015<\/a>) achieves 50.9% accuracy, using a large amount of additional phrase-level annotations, and a much bigger network based on LSTMs and parse trees. As there are 5 classes to choose from, a random system would get 20% accuracy.<\/p>\n Theano is smart enough to move some parts of the processing to the GPU, as long as CUDA is installed and a graphics card is made available. To install CUDA, follow instructions on one of these links:<\/p>\n https:\/\/developer.nvidia.com\/cuda-downloads<\/a> Then, when running your Python script, you need to point Theano to the CUDA installation. I do this by setting the environment variables in the command line:<\/p>\n This command is for CUDA-7.5 in my system. You’ll need to make sure that the paths match the CUDA installation paths in your machine.\u00a0If it works and Theano is using a GPU, the first line that gets printed will explicitly say so. Something like this:<\/p>\n If you don’t get something similar, it probably means Theano is not\u00a0properly hooked up to use the GPU.<\/p>\n At the time of writing, Theano only supports 32-bit variables on the GPU, and this is where the\u00a0floatX=float32 setting comes it. It\u00a0just allows you to set the data type during the execution of the script, without writing it into your code. For example, you can define your vectors like this:<\/p>\n And now\u00a0you can set floatX to be float32 when running the script on a GPU and float64 when running on your CPU.<\/p>\n Finally, if your machine has multiple GPUs, you can control which one is used for the script by setting\u00a0device=gpu0,\u00a0device=gpu1, etc. Based on personal experience, running multiple Theano jobs on the same GPU does not give any advantage, so it’s best to send them to different ones when possible.<\/p>\n Theano provides a command for printing a variable or a function, along with all the required computation, as an image:<\/p>\n When dealing with very simple models, this can give a nice graphical representation. For example, here is a model from our minimal working example:<\/p>\n However, when the models get more and more complicated, the images also tend to get less informative:<\/p>\n <\/p>\n Finally, Theano also provides a useful tool for analysing bottlenecks in your code. Just set profile=True in THEANO_FLAGS, and it will print information about how much time is spent on different operations in your code.<\/p>\n This concludes the Theano tutorial. If you haven’t yet had enough, take a look at the following links that I used for inspiration: <\/p>\n","protected":false},"excerpt":{"rendered":" This is an introductory tutorial on\u00a0using Theano, the Python library. I’m going to\u00a0start from scratch and assume no previous knowledge of Theano. However, understanding how…<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[],"class_list":["post-485","post","type-post","status-publish","format-standard","hentry","category-uncategorized"],"yoast_head":"\n1. What is Theano?<\/h2>\n
![\"CYh2GMnWkAELDTL\"](\"https:\/\/www.marekrei.com\/blog\/wp-content\/uploads\/2016\/01\/CYh2GMnWkAELDTL.jpg\")
<\/p>\n\n
\nTechnically, Theano isn’t actually a machine learning library, as it doesn’t provide you with pre-built models that you can train on your dataset. Instead, it is a mathematical library that provides you with tools to build your own machine learning models. But if you are looking for machine learning toolkits, there are several good ones implemented on top of Theano:<\/p>\n\n
2. Python refresher<\/h2>\n
\n
def f(x):\r\n return 2*x<\/pre>\n<\/li>\n
a = [1,2,3,4,5]\r\na[1] == 2<\/pre>\n<\/li>\n
b = {'key1': 1, 'key2':2}\r\nb['key2'] == 2<\/pre>\n<\/li>\n\r\nc = [i+1 for i in range(5)]\r\nc[1] == 2<\/pre>\n<\/li>\n<\/ul>\n
3. Using Theano<\/h2>\n
sudo apt-get install python-numpy python-scipy python-dev python-pip python-nose g++ libopenblas-dev git\r\nsudo pip install Theano\r\n<\/pre>\n
\nhttp:\/\/deeplearning.net\/software\/theano\/install.html<\/a><\/p>\n\r\nimport theano\r\n<\/pre>\n
4. Minimal Working Example<\/h2>\n
import theano\r\nimport numpy\r\n\r\nx = theano.tensor.fvector('x')\r\nW = theano.shared(numpy.asarray([0.2, 0.7]), 'W')\r\ny = (x * W).sum()\r\n\r\nf = theano.function([x], y)\r\n\r\noutput = f([1.0, 1.0])\r\nprint output<\/pre>\nx = theano.tensor.fvector('x')<\/pre>\nW = theano.shared(numpy.asarray([0.2, 0.7]), 'W')<\/pre>\n
y = (x * W).sum()<\/pre>\n
f = theano.function([x], y)<\/pre>\n
output = f([1.0, 1.0])<\/pre>\n
0.2*1.0 + 0.7*1.0 = 0.9<\/pre>\n
5. Symbolic graphs in Theano (!)<\/h2>\n
y = (x * W).sum()<\/pre>\n
6. Variables<\/h2>\n
x = theano.tensor.fvector('x')<\/pre>\n
\nConstructor,\u00a0dtype,\u00a0ndim
\nfvector,float32,1
\nivector,int32,1
\nfscalar,float32,0
\nfmatrix,float32,2
\nftensor3,float32,3
\ndtensor3,float64,3
\n[\/table]<\/p>\nx = theano.tensor.vector('x', dtype=float32)<\/pre>\n7. Shared variables<\/h2>\n
W = theano.shared(numpy.asarray([0.2, 0.7]), 'W')<\/pre>\n
W.get_value()\r\nW.set_value([0.1, 0.9])<\/pre>\n
8. Functions<\/h2>\n
f = theano.function([x], y)<\/pre>\n
\n
9. Minimal Training Example<\/h2>\n
\r\nimport theano\r\nimport numpy\r\n\r\nx = theano.tensor.fvector('x')\r\ntarget = theano.tensor.fscalar('target')\r\n\r\nW = theano.shared(numpy.asarray([0.2, 0.7]), 'W')\r\ny = (x * W).sum()\r\n\r\ncost = theano.tensor.sqr(target - y)\r\ngradients = theano.tensor.grad(cost, [W])\r\nW_updated = W - (0.1 * gradients[0])\r\nupdates = [(W, W_updated)]\r\n\r\nf = theano.function([x, target], y, updates=updates)\r\n\r\nfor i in xrange(10):\r\n output = f([1.0, 1.0], 20.0)\r\n print output\r\n<\/pre>\ntarget = theano.tensor.fscalar('target')<\/pre>\ncost = theano.tensor.sqr(target - y)<\/pre>\n
gradients = theano.tensor.grad(cost, [W])<\/pre>\n
W_updated = W - (0.1 * gradients[0])<\/pre>\n
updates = [(W, W_updated)]<\/pre>\n
f = theano.function([x, target], y, updates=updates)<\/pre>\n
\r\nfor i in xrange(10):\r\n output = f([1.0, 1.0], 20.0)\r\n print output\r\n<\/pre>\n
\r\n0.9\r\n8.54\r\n13.124\r\n15.8744\r\n17.52464\r\n18.514784\r\n19.1088704\r\n19.46532224\r\n19.679193344\r\n19.8075160064\r\n<\/pre>\n
10. Useful operations<\/h2>\n
\r\n> a = theano.shared(numpy.asarray([[1.0,2.0],[3.0,4.0]]), 'a')\r\n> a\r\na\r\n> a.eval()\r\narray([[1., 2.],\r\n [3., 4.]])\r\n<\/pre>\n
c = ((a + a) \/ 4.0)\r\n\r\narray([[ 0.5, 1. ],\r\n [ 1.5, 2. ]])\r\n<\/pre>\n
\r\nc = theano.tensor.dot(a, a)\r\n\r\narray([[ 7., 10.],\r\n [15., 22.]])\r\n<\/pre>\n
\r\nc = theano.tensor.nnet.sigmoid(a)\r\nc = theano.tensor.tanh(a)\r\n\r\narray([[ 0.76159416, 0.96402758],\r\n [ 0.99505475, 0.9993293 ]])\r\n<\/pre>\n
\r\nc = theano.tensor.nnet.softmax(a)\r\n\r\narray([[ 0.26894142, 0.73105858],\r\n [ 0.26894142, 0.73105858]])\r\n<\/pre>\n
\r\nc = a.sum()\r\nc = a.sum(axis=1)\r\n\r\narray([ 3., 7.])\r\n<\/pre>\n
\r\nc = a.max()\r\nc = a.max(axis=1)\r\n\r\narray([ 2., 4.])\r\n<\/pre>\n
\r\nc = theano.tensor.argmax(a)\r\nc = theano.tensor.argmax(a, axis=1)\r\n\r\narray([1, 1])\r\n<\/pre>\n
\r\na = theano.shared(numpy.asarray([[1,2],[3,4]]), 'a')\r\nc = a.reshape((1,4))\r\narray([[1, 2, 3, 4]])\r\n\r\nc = a.reshape((-1,))\r\narray([1, 2, 3, 4])\r\n<\/pre>\n
\r\nc = theano.tensor.zeros_like(a)\r\narray([[0, 0],\r\n [0, 0]])\r\n<\/pre>\n
\r\na.eval()\r\narray([[1, 2],\r\n [3, 4]])\r\n\r\nc = a.dimshuffle((1,0))\r\narray([[1, 3],\r\n [2, 4]])\r\n\r\nc = a.dimshuffle(('x',0,1))\r\narray([[[1, 2],\r\n [3, 4]]])\r\n<\/pre>\n\r\na = theano.shared(numpy.asarray([[1.0,2.0],[3.0,4.0]]), 'a')\r\narray([[1., 2.],\r\n [3., 4.]])\r\n\r\nb = [1,1,0]\r\nc = a[b]\r\narray([[ 3., 4.],\r\n [ 3., 4.],\r\n [ 1., 2.]])\r\n<\/pre>\n
a[0] = [0.0, 0.0]<\/pre>\n
\r\nc = theano.tensor.set_subtensor(a[0],[0.0, 0.0])\r\narray([[ 0., 0.],\r\n [ 3., 4.]])\r\n<\/pre>\n
11. Classifier Code Example<\/h2>\n
\n
\r\npython classifier.py data\/countries-classify-gdp-normalised.train.txt data\/countries-classify-gdp-normalised.test.txt\r\n<\/pre>\n
\r\nEpoch: 0, Training_cost: 28.4304042768, Training_accuracy: 0.578512396694\r\nEpoch: 1, Training_cost: 24.5186290354, Training_accuracy: 0.619834710744\r\nEpoch: 2, Training_cost: 22.1283727037, Training_accuracy: 0.619834710744\r\nEpoch: 3, Training_cost: 20.7941253329, Training_accuracy: 0.619834710744\r\nEpoch: 4, Training_cost: 19.9641569475, Training_accuracy: 0.619834710744\r\nEpoch: 5, Training_cost: 19.3749411377, Training_accuracy: 0.619834710744\r\nEpoch: 6, Training_cost: 18.8899216914, Training_accuracy: 0.619834710744\r\nEpoch: 7, Training_cost: 18.4006371608, Training_accuracy: 0.677685950413\r\nEpoch: 8, Training_cost: 17.7210185975, Training_accuracy: 0.793388429752\r\nEpoch: 9, Training_cost: 16.315597037, Training_accuracy: 0.876033057851\r\nTest_cost: 5.01800578051, Test_accuracy: 0.925\r\n<\/pre>\n
12. Recurrent functions with scan<\/h2>\n
\r\ndef rnn_step(x, previous_hidden_vector, W_input, W_recurrent):\r\n hidden_vector = theano.tensor.dot(x, W_input) + \r\n theano.tensor.dot(previous_hidden_vector, W_recurrent)\r\n hidden_vector = theano.tensor.nnet.sigmoid(hidden_vector)\r\n\r\nW_input = self.create_parameter_matrix('W_input', (word_embedding_size, recurrent_size))\r\nW_recurrent = self.create_parameter_matrix('W_recurrent', (recurrent_size, recurrent_size))\r\ninitial_hidden_vector = theano.tensor.alloc(numpy.array(0, dtype=floatX), recurrent_size)\r\n\r\nhidden_vector, _ = theano.scan(\r\n rnn_step,\r\n sequences = input_vectors,\r\n outputs_info = initial_hidden_vector,\r\n non_sequences = [W_input, W_recurrent]\r\n)\r\n\r\nhidden_vector = hidden_vector[-1]\r\n<\/pre>\n\n
\n
self<\/code>.create_parameter_matrix<\/code>, definition not shown here) which takes as input the variable name and the shape. This means I don’t need to define the weight initialisation part again each time.<\/p>\n13. RNN Classifier Code Example<\/h2>\n
\r\npython stanford_sentiment_extractor.py 1 full \/path\/to\/sentiment\/dataset\/ > data\/sentiment.train.txt\r\npython stanford_sentiment_extractor.py 2 full \/path\/to\/sentiment\/dataset\/ > data\/sentiment.test.txt\r\n<\/pre>\n
\r\npython rnnclassifier.py data\/sentiment.train.txt data\/sentiment.test.txt\r\n<\/pre>\n
\r\nEpoch: 0 Cost: 25937.7372292 Accuracy: 0.285814606742\r\nEpoch: 1 Cost: 21656.820174 Accuracy: 0.350655430712\r\nEpoch: 2 Cost: 18020.619533 Accuracy: 0.429073033708\r\nTest_cost: 4784.25137484 Test_accuracy: 0.388235294118\r\n<\/pre>\n
14. Running on a GPU<\/h2>\n
\nhttp:\/\/www.r-tutor.com\/gpu-computing\/cuda-installation\/cuda7.5-ubuntu<\/a><\/p>\n\r\nLD_LIBRARY_PATH=\/usr\/lib:\/usr\/local\/cuda-7.5\/lib64 THEANO_FLAGS='cuda.root=\/usr\/local\/cuda-7.5,device=gpu,floatX=float32' python mycode.py\r\n<\/pre>\n
Using gpu device 0: GeForce GTX 780<\/pre>\n
x = theano.tensor.vector('x', dtype=config.floatX)<\/pre>\n15. Drawing the computation graph<\/h2>\n
\r\nf = theano.function([x], y)\r\ntheano.printing.pydotprint(f, outfile="f.png", var_with_name_simple=True)\r\n<\/pre>\n
![\"Printed](\"https:\/\/www.marekrei.com\/blog\/wp-content\/uploads\/2016\/01\/f-300x246.png\")
<\/p>\n![\"Printed](\"https:\/\/www.marekrei.com\/blog\/wp-content\/uploads\/2016\/01\/logreg_pydotprint_train2.png\")
<\/p>\n16. Profiling<\/h2>\n
THEANO_FLAGS='profile=True' python minimal_working_example.py<\/pre>\n
![\"Example](\"https:\/\/www.marekrei.com\/blog\/wp-content\/uploads\/2016\/01\/profiling.png\")
<\/p>\n17. References<\/h2>\n
\nOfficial Theano homepage and documentation<\/a>
\nOfficial Theano tutorial<\/a>
\nA Simple Tutorial on Theano\u00a0by\u00a0Jiang Guo<\/a>
\nCode samples for learning Theano\u00a0by\u00a0Alec Radford<\/a><\/p>\n