The lure of machine learning isn’t always about big new features; often, what it does best are small tweaks that subtly improve user experience. So it is with Twitter’s use of neural networks to automatically crop picture previews to their most interesting part.
The company’s been working on this tool for a while, but described its methods in detail in a blog post yesterday. It’s an interesting little read, with ML engineer Lucas Theis and ML lead Zehan Wang explaining how they started just using facial recognition to crop images to faces, but found that this method didn’t work with pictures of scenery, objects, and, most importantly, cats.
Their solution was “cropping using saliency” (saliency here meaning whatever’s most interesting...
