Can deep learning help you find the perfect girl?

One of the first things I did when I moved to Montreal was installing Tinder. For those of you not familiar with the online meat market, Tinder is a dating app showing nearby users you can like or dislike by their profile pictures. After using the app for a while, I discovered that although I consistently disliked girls with a lot of piercings and tattoos (no offence, just not my type), the app kept showing me these profiles. It surprised me that Tinder did not use my history of swipes to learn what type of girls I like.

This observation made me think: can a computer learn to which girls I'm attracted? I have tried by labeling over 9K profile pictures on Tinder and using deep learning, the latest revolution in artificial intelligence. In this blog post, I will provide a high-level view how I used these techniques to predict my Tinder swipes. You can find the technical details in this paper, which was accepted for the ICML deep learning workshop.

Machine learning

Computers have been around for many decades, but there still exists a lot of problems that the most powerful computers struggle to solve, yet we (as human beings) can do without effort. You can think of detecting faces in pictures, recognizing speech and translating text. The problem is that although we perform such tasks easily, we cannot explain how we do it. That means, we can not come up with a set of rules that can be transformed into a computer program.

Is predicting attractiveness from a profile picture such a task? Yes, it is simply too difficult to specify a set of rules to who you're attracted. For some of you this might sound counter-intuitive because you can specify a clear list of traits e.g. blue eyes, blonde hair, etc. However, even if you could do that, you have to believe me that it is too complex to construct a program by hand that detects such characteristics in images.

On the other hand, if I show you a picture of a girl/boy you can decide in a split of a second whether you like her/him. This suggests a better strategy: we let the computer program itself by showing it profile pictures of girls you do or do not find attractive. We call this machine learning because the program learns a set of attractiveness rules from these profile pictures.

Deep learning

Deep learning is a particular class of learning algorithms that has recently lead to a breakthrough in image recognition (and many other artificial intelligence problems). If you want to see yourself how well these techniques work nowadays, I encourage you to upload a picture to

In deep learning, the structure of the program is an artificial neural network which is inspired by a biological neural network i.e. the brain. Neural networks are called deep when they consists of several layers of connected neurons as shown in the figure above. Each neuron is connected to a set of neurons in the previous layer. The strength of those connections (shown in red) determine to what image pattern the neuron responds. You can think of it as an adjustable filter, only responding when it sees a particular pattern.

The connection strengths of the network are randomly initialized, meaning that all neurons are just random filters. Therefore, when we feed an image to the network (by inputting the intensity of its pixels), only a few neurons in the first layer will respond, which in turn let some random neurons in the next layer to be active (and so forth till the output). As a result, the output of the network will be random like or dislike.

Therefore, in order to make good predictions, we need to train the network. We do so by inputting a profile picture and propagating it through the network to get the output. If the network's prediction is different from the desired output (like or dislike), we adjust its filters (i.e. its connection strengths) to make the prediction more correct. How we exactly adjust the filters is rather difficult and outside the scope of this blog post, all you need to know is that we can do it. The training phase is nothing more than repeating this process for many examples, and then hope that the network predicts well for profile pictures it has not seen yet.

After training, we often observe that the neurons in the first layer respond to edges and color blobs, while the next layer uses these observations to detect more complex patterns such as eyes and ears. The last layer might detect complete faces or piercings and tattoos (which is important to predict my preferences).

Collecting data

Neural networks work especially well when we provide them a lot of labeled examples. So, all I needed to do was intensively use Tinder for some time.

But how do you save the liked and disliked pictures from the Tinder app? The application is communicating with a server to get new profile pictures, and by inspecting how the app did this communication, I could build a program on my laptop that mimics the tinder app (but saves the pictures). You can see the program at work in the screenshot. I have to say that the real Tinder app was a bit more convenient because it can be used anywhere and anytime. Now I really had to sit down and make time to swipe profiles. Nevertheless, I managed to label 9364 profile pictures in less than two months.

Almost 10K profile pictures sounds a lot, but in practice it was not enough to successfully train a deep neural network. What happened was that the network worked very well for the pictures it was trained on, but did not generalize to new profile pictures. Looking at the images, I could see that there was a lot of variation in the profile pictures (compared to the clean datasets we work with in academia). One way to account for those variations is to collect more data, but I didn't want to wait half a year before I had labeled enough profile pictures (or spent my whole day using Tinder..).

Instead, I focused on another way to overcome this issue: we can use another neural network that is successfully trained for a related task (for which there was enough data). The lower-layer filters learned by that network are recognizing patterns that might be useful for our task. We simply copy the trained network, and then only adjust the last layers to predict my attractiveness preferences.

I thought it would be a good idea to use a network to predict gender from profile pictures for two reasons. First, it is easy to collect data for this task. I scraped over 400 000 (!) male and female profile pictures from OKCupid.com in a couple of weeks. Second, the filters extracted by this network could be useful to predict my preferences, since I am probably more attracted to girls that do not look like men.

The data never lie..

Now that I had collected the data, I had to face the facts about my own Tinder behavior:

  • It was very confronting that I liked more than half (53%) of all profile pictures. Maybe I'm not that picky, but I also believe that Tinder shows you the popular profiles more frequently. There is a clear incentive for Tinder since you keep using the app when you hope to match with pretty girls. I can also imagine that more attractive people are using Tinder, and my swiped profile pictures are thus not representatives of the whole population (I think it would be weird if I was willing to date more than half of the population).
  • "What was I thinking when I liked this picture??" This feeling popped up several times when scrolling through the labeled pictures. It was tempting to do another round of labeling to filter out this unwanted likes. I realized that the problem is more general: for some pictures I do not have a clear opinion, and my mood determines the label. So, instead I estimated how much "noise" entered the labeling process. I relabeled a 100 random profile pictures a couple of weeks later and compared the results with the original labeling. I judged 12 pictures differently.
  • From the 12 errors I made, we could assume that I have guessed 24% of the time (because 24*0.5=12). You have to realize that this makes it difficult for any model to learn my preferences. If you are not consistent in the labeling, you are confusing the network what are important image patterns to learn.

The results

I'll spare you the details of processing all this data, and simply summarize the results:

  • The neural network trained on profile pictures from OKcupid predicted 92.7% of the time correctly if there was a male or female in the image. It took about 3 days to train this network on one of the fastest graphical cards today. It achieves near human performance because I achieved myself around 95% on this dataset. There were a lot of pictures with both males and females in the picture, and also encountered some pictures without somebody in the picture (apparently there are people who put a picture of a plate of food as their profile picture).
  • Given the success of the gender network, I had good hope that this network could be used to predict my preferences. Unfortunately, this wasn't the case, this network only achieved 61.5% accuracy.
  • However, I used another network that was trained (not by me) to distinguish all kinds of animals, plants and vehicles. Somewhat surprising, predicting attractiveness using this network worked quite well: 68.1% accuracy.

Almost 70% does not sound that impressive, right? But how well could someone else predict to which girls I am attracted?

My co-author Jason Yosinski tried it. He 'trained' by first looking at all 50 dislike and 50 like pictures side by side, scrolling through them all a few times. Then the training set was shuffled, pictures were displayed one a time, and he produced a label prediction after each photo. The correct label was shown after each picture so that he could learn from his mistakes. See his study program in the screenshot above. He achieved 86%, 82%, 88% and 88% on 4 training rounds.

He then predicted the labels for 100 unseen profile pictures (the same ones as in my consistency test). He made 24 errors, and thus achieved 76% accuracy. Given that's also hard for other people to predict my preferences, the 68% is not that bad!

The future

The techniques I have used are not tailored to my preferences, and could be used for other people whenever they provide their labeled profiles pictures. In fact, deep learning is known to be working much better with bigger datasets, so I am pretty confident that this approach would be even more successful when I train a neural network on data gathered from many people (e.g. when I have access to all data from Tinder). As far as I know, the most popular dating websites (like OKCupid.com) still use questionaires to suggest potential partners. I think the success of Tinder has shown that people care about somebody's appearance, and with the rise of deep learning we now have a tool to actually incorporate this kind of information in matching algorithms.

The time that you were predetermined to marry one of the 50 girls/boys of your town is over. New technologies will continue to change the way how we find our romantic partners. Although randomness sometimes determines love, we should embrace the great opportunities in machine learning to help anyone looking for a soulmate!

Can deep learning help you find the perfect girl?的更多相关文章

  1. The Brain vs Deep Learning Part I: Computational Complexity — Or Why the Singularity Is Nowhere Near

    The Brain vs Deep Learning Part I: Computational Complexity — Or Why the Singularity Is Nowhere Near ...

  2. Deep Learning 12_深度学习UFLDL教程:Sparse Coding_exercise(斯坦福大学深度学习教程)

    前言 理论知识:UFLDL教程.Deep learning:二十六(Sparse coding简单理解).Deep learning:二十七(Sparse coding中关于矩阵的范数求导).Deep ...

  3. Deep Learning Enables You to Hide Screen when Your Boss is Approaching

    https://github.com/Hironsan/BossSensor/ 背景介绍 学生时代,老师站在窗外的阴影挥之不去.大家在玩手机,看漫画,看小说的时候,总是会找同桌帮忙看着班主任有没有来. ...

  4. Coursera Deep Learning 2 Improving Deep Neural Networks: Hyperparameter tuning, Regularization and Optimization - week1, Assignment(Gradient Checking)

    声明:所有内容来自coursera,作为个人学习笔记记录在这里. Gradient Checking Welcome to the final assignment for this week! In ...

  5. Background removal with deep learning

    [原文链接] Background removal with deep learning   This post describes our work and research on the gree ...

  6. 课程二(Improving Deep Neural Networks: Hyperparameter tuning, Regularization and Optimization),第一周(Practical aspects of Deep Learning) —— 4.Programming assignments:Gradient Checking

    Gradient Checking Welcome to this week's third programming assignment! You will be implementing grad ...

  7. Rolling in the Deep (Learning)

    Rolling in the Deep (Learning) Deep Learning has been getting a lot of press lately, and is one of t ...

  8. 深度学习Deep learning

    In the last chapter we learned that deep neural networks are often much harder to train than shallow ...

  9. 【CS-4476-project 6】Deep Learning

    AlexNet / VGG-F network visualized by mNeuron. Project 6: Deep LearningIntroduction to Computer Visi ...

随机推荐

  1. 机器学习库shark安装

    经过两天的折腾,一个对c++和机器学习库的安装都一知半解的人终于在反复安装中,成功的将shark库安装好了,小小纪念一下,多亏了卡门的热心帮忙. shark的安装主要分为以下几个部分: (1)下载 s ...

  2. redis在centOS的安装

    1.安装tcl支持 yum install tcl 2.安装redis我们以最新的2.8.9为例 $ wget http://download.redis.io/releases/redis-2.8. ...

  3. 射击的乐趣:WIN32诠释打飞机游戏源码补充

    打飞机游戏源码补充 从指定位置加载bmp并显示到对话框. , TRUE);, , LR_LOADFROMFILE);          {         BITMAP bmpinfo;        ...

  4. Java中的数组问题

    java.util.Arrays  This class deals with 'real' arrays in java, in the form of T[]. Thus it doesn't d ...

  5. Wpf学习之路……

    Wpf学习之路-- Wpf是 .net中一门做winform的技术,和传统的winform的区别在于: 1.         原来的winform就是拖控件,而wpf的控件都死自己拿标记语言(xaml ...

  6. [C#]使用Quartz.NET来创建定时工作任务

    本文为原创文章.源代码为原创代码,如转载/复制,请在网页/代码处明显位置标明原文名称.作者及网址,谢谢! 开发工具:VS2017 语言:C# DotNet版本:.Net FrameWork 4.0及以 ...

  7. H - Rescue the Princess ZOJ - 4097 (tarjan缩点+倍增lca)

    题目链接: H - Rescue the Princess  ZOJ - 4097 学习链接: zoj4097 Rescue the Princess无向图缩点有重边+lca - lhc..._博客园 ...

  8. web site optimization

    @ 如果有很多图片(比如web服务器的页面上有多个小图片),通常是没有必要记录文件的访问时间的,这样就可以减少写磁盘的I/O,这个要如何配置 @ 首先,修改文件系统的配置文件/etc/fstab ,然 ...

  9. PowerShell使用教程

    一.说明 1.1 背景说明 个人对PowerShell也不是很熟悉,开始的时候就突然看到开始菜单中多了个叫PowerShell的文件夹,后来一点就看到某个教程视频说PowerShell很厉害但也没怎么 ...

  10. Linux下的消息队列

    文章链接:https://blog.csdn.net/qq_38646470/article/details/80169406