FWIW in 2016 I was at an ML team at Apple that had been shipping production neural networks on-device for a while already. At the everyone used an assortment of random tools (Theano, Torch, Caffe). I worked on an internal tool that originally started as a Theano fork but was closer to a modern-day Tensorflow XLA (and has since been axed in favor of Tensorflow for most teams).