(Part 2) Best products from r/MachineLearning

check out Bengio's paper if you haven't yet. There's a few really cool pieces, but the most relevant I think... the first chunk of the paper looks at a really simple two discrete random variable system, and posits two possible causal models: X -> Y and Y -> X. The thrust of that part of the paper is basically that fitting both those causal models is equally expensive, you've got the same number of model weights after all. The magic happens when you change p(x) for X -> Y = p(x)p(y|x) and do a transfer learning cycle on this new distribution. For the 'wrong' model, you have to refit every single model weight, because the structure of the model isn't captured in a way that separates that causal connection properly, it's distributed through the model instead. For the 'right' model though, he links to another paper showing that the gradient is zero for all the already correct parameters, so you end up just changing N of the underlying change instead of the full O(N^2) parameters of the model. He's got a graph showing convergence for the 'right' and 'wrong' model on the transfer learning objective.... both converge to the same spot, but the difference between the number of samples needed to converge for both is really, really huge. The 'wrong' causal model takes massively more samples to converge. From your very first observed example on the transfer dataset too, the sparseness of that gradient on a transfer objective for the 'right' model was how you could distinguish the correct model even. Your point about parameters needing an update being kept small is right I think... The question though is how to make sure that's reliably the case in general. There's some really cool stuff in disentangled representation learning for RL too I think... I don't know. I guess at this point I'm sold that for a model (in general) to properly isolate the various moving parts of the system instead of representing it through the whole thing in a giant mess, will require a new approach to learning.

Course, that doesn't mean you can't get some of that separation just by carefully controlling the training set and order you see them in. You're completely right that there's some really cool generalization power that can come up with the right training protocol (Learning to Make Analogies by Contrasting Abstract Relational Structure, Emergent systematic generalization
in a situated agent, and ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness come to mind) but I think as long as we're just using our current dumb AI systems with a carefully manicured training protocol, we're missing a huge piece of the puzzle. We'll always need some level of curriculum management of course (humans obviously benefit from the right training material too) but I'm convinced enough that the ability to explicitly form a proper disentangled representation of the environment is key that I'm heading in that direction in my studies. Guess we'll see in a decade whether or not I regret my focus, haha.

And yeah, I think there's a ton of insight to be gained from studying biological consciousness. I actually started scrapping my way into that six months ago or whatever on the side. I'm currently 600 pages into Kandel's beastly 1,700 page 'principles of neural science'. All that's just preliminary biology stuff, but there's some really cool looking books I want to hit when I get a little farther in. this book especially is one I'm excited to hit next when I'm done with Kandel, it looks like it's doable without a ton of background in neuro, sounds like you might enjoy that one too. Beyond that, check out Jeff Hawkins 'on intelligence' (and the research of his group if you're interested in what he has to say... cool stuff there about cortical column functioning as a building block of cognition) and Christof Koch's 'Consciousness: Confessions of a Romantic Reductionist'. Both of those two books are just for lay people, so you could blow through them real quick to get a flavor of their ideas, but that last one especially... Koch seems to be involved in the only (that I've found so far) falsifiable model of consciousness. It has to do with information integration between disparate parts of a system... really cool sounding ideas, but the math is absolutely beastly in the theory itself, haha. I'm not equipped yet to weather it, but it seems like there's some really important ideas there too from what little I've grasped so far.

Anyway, yeah... totally agree. Might as well take inspiration from the one working example of a strong AI system we have access to, haha.

As I understand it you want to learn more about Machine learning overall and you want to use the task as an example.

Firstly beside all the nice books and the hype in AI in the last 10 years; It is not fundamental different from the knowledge we had 40 years ago. Most here is noise by a generation born after the AI winter. Some of us who stayed in the than toxic field tried to preserves as much as we could. Right now most you read is noise and very specific application use-cases and "optimization in the saturation" and simply just more computing power.

Of cause it is possible to use techniques/models to solve your problem, which are today attributed to AI and ML. It just depends on your point of view which one you want to use. AI is decision oriented. ML is representation oriented, meaning in all parts of reality: time, space and information.

I will help you with the analysis and can present some options. I will keep it short and superficial.
I will use the Top Down approach.

We have:

• Environment Grid 2D, Cartesian Coordinats
  
• passive entitys with defined behavior
  
• agent as only active entity
  
  All data can be represented in discrete linear dependent units. There is no probabilistic or fuzzy states or relations without forcing it.
  
  -> Representation A (Ball as entity/agent as entity) Solution as agent-based unsupervised RL :
  
  
• Set of all balls; as action queue or what ever, even liked list are posible
  
• Agent = Sense+will+possible actions+possible reaction at t+1 = t+1
  
• Sense is the set of balls & the pain/reinforcement signal
  *Will is your reward-function
  
• possible reactions are reactions of the environment to your actions, the possible stands in for a "knowledge-engine" of any kind
  
• actions are to Strategies your systems can take based on a control decision
  
  From your Task it is not clear if the agent can perform multiple actions per time frame or any limited number. Or if the actions are limited to certain locations. This will define how you want to store your action models and state perception.
  
  My guess is, this is the point you are asking about; What kinds of "knowledge-engine" are there and what could I use. If so there are alot of options.
  
  
• classic control theory (AI)[t->t+1]
  like: ID ball_move_next = popfirst.sortmax.urgentscore(ID_set,subscore_Y,subscore_X);
  which will not use the reinforcement function.
  
  
• sequence based(ML -> AI)[t+(t+1)' -> t+1]
  in this case sequence footed on the time. With every step being dependent from on a discrete environment. The focus here is the environment.
  
  you could use Markov chains, NN, even a ringbuffer with a memory field and a decision hierarchy. All the nice toys that are "in" at the moment. Seq2seq would be a "best path" solution which could be checked if balls would collide... and so on.
  
  
• "suffering agent"(ML->AI)[t -> t+1]
  In this scenario the "inner world" of the agent is focus. Based on the idea of frustration-tolerance and reward-delay. Here the "knowledge-engine" tweaks the reward and the reinforcement signal, as well as storing the IDs of the entity and its "history" with the agent. And after the reward the "history" is compared to other histories, this will form action preferences.
  
  How you solve it depends on your machine power, knowledge, preference.
  
  -> Representation B (state map) :
  
  You focus here on the environment, like in physical simulations(FEM). The interaction is between neighboring states.
  
  The AI part is the description of the states in relation to the actions and the action itself. The ML part is a dm(decision matrix), mapping which state invokes which action. This could be grouped in the cellular automate family.
  
  The dm is like the "knowledge-engine" in the other example. So you can use a wide variety of optimization systems. A big plus for this way of solution would be that you can compute each set of neighbors independent, like the Input in convolution nets.
  
  
  I am not sure if this can push you to an experiment you are happy with to try. This here is very superficial and simplified.
  
  To learn a systemic view I advice you to stop learning the "tools"(like NN,RL,UL and so on) and start with classic cybernetics/AI. Like Neumann, shannon,von Foerster,John McCarthy,Jay Wright Forrester even Lems Summa Technologiae could help. First you should learn to think broad. this book is very easy to understand, which might help as well. To just focus on the opinions what ML/AI is today and study this tools wouldn´t bring you forward, its like knowing only how to saw, while wanting to build a house.

Pick up a numerical analysis book of some sort. I learned out of Sauer, "Numerical Analysis" which I wasn't incredibly happy with and so I can't recommend it, but it covered the basics with example code in Matlab. Someone else here can probably recommend a better numerical methods textbook.

Some great free resources:

• Probabilistic Programming & Bayesian Methods for Hackers (hard copy: Amazon)
  
• Barber, "Bayesian Reasoning and Machine Learning" (hard copy: Cambridge University Press
  
  Other than that I don't know of any resources that could help with implementing machine learning algorithms; it mostly draws from numerical linear algebra / optimization, and the implementation details are left to the reader to grapple with.
  
  For me, it works best to buy a good theory book (Koller & Friedman!) or read a good paper, and look around for blog posts written about a topic of interest that come with examples and sample code. It's unfortunate that everything is so disorganized this way but that's how it seems to be. I'd be happy to learn that I'm wrong here, though :)
  
  Some blogs I can recommend:
  
  
• Mathieu's Log
  
• Edwin Chen
  
• Math ∩ Programming
  
• Randal S. Olson
  
• Almost Looks Like Work
  
  Hope this is helpful! This is basically how I've been learning ML on my own for work.
  
  
  

> Or can someone shed some light on what they're discussing and what this paper is proposing?

1. Consciousness (atleast, consciousness(es) that we are familiar with) seems to occur at a certain scale. Conscious states doesn't seem to significantly covary with noisy schocastic activities of individual cells and such; rather it seems to covary at with macro-level patterns and activities emereging from a population of neurons and stuffs. We are not aware of how we precisely process information (like segmenting images, detecting faces, recognizing speeches), or perform actions (like precise motor controls and everything). We are aware of things at a much higher scale. However, consciousness doesn't seem to exist at an overly macro-level scale either (like, for example, we won't think that USA is conscious).
   
   
   
2. The authors seem to think that the reason consciousness exists in this scale because of the property of 'non-trivial information closure'. A system is informationally closed if the information flow from environment to the system is 0. A trivial case of information closure is when the system and the environment is pretty much independent. For the authors, the degree of consciousness is instead associated with the degree of closure in non-trivially closed informational systems. What is 'non-trivial information closure'? - in this case, even though the environment at time t (E_t) plays a role in the formation of the system state in time t (Y_t), Yt encodes enough information about itself , the environment, and the 'environment's influence on itself', that it is possible for the system to predict much of (not necessarily everything) Y{t+1} just on the basis of Y_t alone, without accessing E_t.
   
   
   2.5) Rejection of 'trivial information closure' helps a bit with bounding conditions. We can think of an aggregate of informationally closed system as a informationally closed system, but we wouldn't think that a mere aggregate of potentially 'conscious' minds are together having a single unitive consciousness. Since trivial information closure doesn't contribute consciousness according to their hypothesis, adding independent closed systems to another system would not change the degree of consciousness of either. This may also have some relationship with the idea of integration in IIT (Information Integration Theory).
   
   
   
3. (2) can explain why consciousness seem to be associated with a certain scale. It is difficult to make prediction by modeling all noisy schocastic neural-celllular-whatever activities. Prediction are easier if essential informations (including ideas of causation, and such) of the environment are modeled at higher 'coarse-grained' scale (see (1)) (more at the level of population than at the level of samples).
   
   
   
4. You may now wonder, even if predictability from self-representated states can exist in a certain scale which happens to be seemingly associated with consciousness, it's not clear why predictibility is necessary for consciousness, nor it's very intuitive that our degree of consciousness depends on predictibility. For that I don't have any clear answers. Intuitively, most of our conscious experiences does seem to be laden with immediate expectations, and anticipations - even if we don't always explicitly notice it. The so-called 'specious present' may always represent immediate past as retention and immediate potential future as anticipation. But besides that, this framework can have other intuitive properties, like for example, following this framework, high-level contentful consciousness must have a much richer representations (of self and environmental information) with a more complex model that has higher predictive prowess - which would need a more complex neural substrate - which seems to affirm the intuition that 'higher consciousness' would correlate with more 'complex stuffs'. It can also explain differences in conscious and unconscious processing. For example, it can explain blindsight (where people report that they are blind - not conscious of visual information; but behave in a manner that shows evidence that they have some access to visual information) by saying that in this case, the environmental visuation information is more directly associated with actions and such; it is not internally representated in a rich state at a coarse grained level offering predictibility - thus people with blindsight are not conscious of their 'sight'.
   
   
   
5. 'predictions' seems to be the central part of the paper, however it still seems to be lacking in intuition about why. However, there is a decent chunk of literature in cognitive science and stuff related to the relationship with predictive processing and cognition. PP, Prediction Error Minimization and such are recent hot topics in cognitive science and philosophy. These line of works may or may not better support the paper. This paper is aware of the works and discusses it close relationship with them. ICT seems to extend upon PP in distinguishing unconscious predictions, and conscious predictions, and incorporate the idea of scale and the relationship of consciousness and coarse-graining. I don't have much of a background about PP, but works of Andy Clark may be good introductory materials: (For example) https://www.amazon.com/Surfing-Uncertainty-Prediction-Action-Embodied/dp/0190933216/ref=sr_1_1?keywords=andy+clark&qid=1570248756&s=books&sr=1-1
   
   I cannot personally vouch for the book, but Andy Clark is one of 'big guys' in the field; so he can be a pretty reliable source.
   
   
   
   
6. ICT seems to work well with some of the other theories of consciousness too (Global Workspace Theory, IIT, PP), which the authors discuss about in the paper. It seems to fill in some gaps of those theories. But I am not very qualified to judge about that.
   
   
   _____
   
   
   About background materials. It seemed pretty readable to me without much of a background. For statements about neural activties, I am just taking their words for it, but the citations can be places to look. You can find more about phenomena like 'blindsight' from googling, if you weren't already aware of it. As opposed to the recommendations made by the other redditor, I don't think it has much to do with anything related to the hard problem of consciousness (Nagel's Bat or Chalmer's zombie) at all and you don't need to read them for this paper - though they can interesting reads for their own sake and can help better understanding the potential limitations - but these work goes on a more philosophical direction not quite related to the scope of the paper. The equations may have some relation with information theory (again the citations may be the best bet for better background). PP seems to be most closely related to the paper with the idea of predictability being on the center. So that may something to explore for background. IIT can be another background material for this: https://journals.plos.org/ploscompbiol/article?id=10.1371/journal.pcbi.1003588
   
   https://www.iep.utm.edu/int-info/

Having done several projects in both image classification and NLP, I can offer some advice:

In long term, owning a GPU is cheaper because there is little cost over time.
For short-term projects renting GPUs (e.g. AWS) is cheaper because there's no upfront cost.

I personally own 2 GTX 1080s with 8GB VRAM each.

For some of the bigger models you will need 6-8GB of VRAM, you can get 12 if you have money for a TITAN.
Much less will severely limit your work in my experience, since Batch Sizes below 8/16 become a lot less stable.

Renting Compute Power

If you have a short project, or are just getting into it, AWS (or Google Cloud) can be a useful resource, but managing VMs is an overhead you have to consider. I have no first-hand experience with AWS/GC (I heard drivers are a pain), but i've worked with MS Azure in the past.
Working with VMs is okay, but you have a little more immediate feedback loops when working on a local machine using jupyter or something similar.
Also, for me working with pay-per-time instances is unnerving because I know my money is ticking down somewhere in the background and it hinders me from experimenting some more. Your mileage may vary.

Building a Desktop Machine

This is the route I personally went with. I have a Desktop machine at home, that I can boot (Wake-on-Lan with RasPi) and control (Jupyter/RDP) remotely from my laptop, or just use directly when I'm at home.
One of the benefits is not having to transfer/synchronize Code and Data between different devices, along with others mentioned before.
Just about any good "Gaming" PC will do very well for Deep learning applications, provided you have a CUDA-capable (i.e. Nvidia) GPU.
The downside to this approach are as follows:

1. You have to build/conceptualize it yourself. The folks at /r/buildapc and /r/buildapcforme can be helpful, and if you don't know what you're doing you can buy a pre-built system. I heard NZXT's gaming-focused Prebuilt service is pretty cheap compared to the current market.
   
2. The current PC hardware market is bad for building a PC right now because a) GPUs are expensive due to Cryptocurrency miners. b) RAM is expensive due to industry shortages.
   
   Also some notes for building a system:
   
   

• Make sure you have enough RAM, or fast storage for your Datasets. If you can fit your Data into ram, slow hard drives are fine, but if you need to load data continuously, invest in a good (ideally NVME/Optane) SSD or your I/O might take up 90% of training time.
  
• If going with a multi GPU setup, or planning to add a second GPU later, make sure your Power supply can handle it. Rule of thumb: 300W for the system +250W per GPU should be more than enough. on a related note, make sure you have enough PCIE lanes for the GPUs (+ some for storage)
  
• "Gaming" branded GPUs are good as well. Doesn't need to be a Founders Edition or Titan (because they are usually more expensive)
  
  

  Using a Laptop
  

  This is an option some people choose, It's not my preferred one.
  Basically you have to choose between two options:
  
  

• small, portable, long battery life, but very bad performance due to none or low-end GPU. Examples: Asus, Acer
  
• bulky, heavy, short battery life. ~80% performance of a desktop system with similar parts (mainly for cooling reasons). Needs a power cord at all times. Basically a Desktop you can carry around. Usually very expensive. Examples: Gigabyte Laptop with 2 GTX 1070, Asus Version with 1 GTX 1080
  
  
  
  Edit: This is my first gold, wow! So this is what it feels like. Many thanks to the generous stranger!

Honestly, I would say you are already qualified. I work in this field. I have my MS in Applied Math, was a ML research assistant in grad school, and have a couple years of experience in software engineering.

Software engineering skills are pretty much the only thing that you may lack you would want to work on, however a lot of that you can learn on the job.

A company would probably let your inexperience in software engineering slide because you are very strong on the data analysis / ML side and you have demonstrated you know how to code. The main thing you need to do is work WITH software engineers on integrating your work with a product.

Some skills/technologies you may want to know about that haven't been mentioned as far as I know :

1. SQL -> e.g. MySQL, Transact-SQL
   
   
2. Java -> Hadoop is written in Java
   
   
3. If you end up working at a web company, PHP/Javascript/HTML are useful to know.
   
   
4. Of course, Linux/Unix command line. Very important.
   
   
5. Python -> You mentioned it but I'd say start using it more. Check out iPython Notebook as that might be something that catches your interest.
   
   >edit: Thanks a ton for the advice so far, the consensus seems to be start learning how to deal with big data using Hadoop or some other similar app and learn some software engineering, however that part seems a little ambiguous still.
   
   Learning software engineering is basically learning best practices and design patterns. The goal is to make clean code and avoid hard to find/fix bugs as well as to use the best design paradigm for your problem. "Clean code" means "readable" and "maintainable". Readable code is code that any other software engineer who has never seen it before can pick up and understand reasonably quickly. Maintainable code is code that is relatively easy to refactor, make changes to, add features, and fix bugs within. "Clean" code saves a lot of man hours down the road.
   
   For an example of software design, in programming a video game OOP class inheritance isn't always the best thing to use because it can lead to a confusing, non-intuitive tree of classes. Someone invented the "component entity system" to address the problem.
   
   Read about this, as it's a fairly intuitive example of "Software design", i.e. using the best approach for a particular problem.
   
   Writing clean, well-designed code comes with experience, but there are some books on it.

The following isn't that organized, but hopefully it's kind of helpful.

You haven't really described how you're going to do stepwise selection or what you're going to do with the output. There are different criteria for judging the models such as AIC and BIC (which are mainly intended to help in choosing a predictive model) and there is step-wise selection based on p-values. Any p-values that come out of any of these procedures are not going to be valid. Much of the criticism of step-wise regression has been because of this fact. Applying a Bonferroni correction seems pretty ad-hoc here. But it is likely too conservative, and that may be why your advisor feels comfortable suggesting it.

Three prominent alternative approaches to this sort of problem are:

1. LASSO type stuff. Here's the original paper, which includes some comparison to subset selection methods like step-wise regression: http://statweb.stanford.edu/~tibs/lasso/lasso.pdf. There are a million variations on the LASSO which have subsequently come out and a ton of theory. A big benefit of the LASSO (and subsequent variations) is that they do shrinkage in addition to subset selection. What finally comes out of a step-wise routine is a model fit with least squares, so the coefficients will be too large. Read some about shrinkage if you're not familiar with this concept. If you plan on using this model for prediction or you actually care about your coefficient estimates in addition to the selection, then this really matters. I haven't read much about it, but there is a recent paper which may be interesting to you: http://statweb.stanford.edu/~tibs/ftp/covtest.pdf
   
   
2. FDR and empirical Bayes. I don't know a ton about this area, but this is another body of methodology which was motivated by the sorts of large scale problems encountered in genetics. This is probably a good book to look at: http://www.amazon.com/Large-Scale-Inference-Estimation-Prediction-Mathematical/dp/110761967X and here's an article on the topic, also by Efron: http://statweb.stanford.edu/~ckirby/brad/papers/2004Selection.pdf
   
   
3. Bayesian approaches. There is a lot of work on shrinkage priors which are good for shrinking effects which are likely null to be very close to zero without shrinking large effects. This doesn't set coefficients exactly to zero, but it comes with uncertainty intervals built in. There has been work lately on frequentist properties (posterior concentration) of these models. See this blog post for a readable discussion and references: https://statblog.net/2013/07/27/shrinkage-priors-computational-vs-statistical-efficiency-2/ Note that the discussion is often framed in terms of the "normal means problem" but you can just think of theta as your regression coefficients.

If you are interested in that stuff you should check out the book The Art of Fermentation by Sandor Katz if you haven't. It's got a lot of good info in it.

I too love what people like Beyond Meat are doing, and I think it's definitely a critical part of food in the future. I'm working on my masters and just doing consulting gigs in the mean time, but I would love to work for a company doing stuff like that.

In terms of software in the food space I think that there is going to be a lot of application in food processing automation. It's something that is a difficult problem to solve, especially when taking manual dexterity of robotics into account, plus cost. Software wise I think it would be really cool to apply image recognition to every day items that need to be processed (take a potato for example) and be able to properly assess size and shape and create instructions for cutting (diced, sliced etc.) for an automated system. Basically just a more human eye for an automated kitchen worker.

In that kind of respect I think we are much further ahead software wise than we are hardware wise, but I feel like that is going to change rapidly over the next 5-10 years as more quick service restaurants push towards cutting costs, of which labor is almost always the highest.

This is my first time reading this page and I am quite the amateur programmer.

I am an Assistant Professor in Criminal Justice; however, my passion is quantitative methodology and understanding big data.

I had a great opportunity to spend a summer learning Bayesian at ICPSR, but to be honest some of the concepts were hard to grasp. So, I have spent the greater part of the past year learning more about maximum likelihood estimations and Bayesian modeling.

I am currently reading The BUGS Book and [Doing Bayesian Analysis] (https://www.amazon.com/Doing-Bayesian-Data-Analysis-Tutorial/dp/0123814855/ref=sr_1_fkmr1_3?s=books&ie=UTF8&qid=1519347052&sr=1-3-fkmr1&keywords=bayesian+anaylsis+bugs).

I regularly teach linear modeling at both the undergraduate and graduate level. Lately, however, I have become interested in other techniques of prediction such as nearest neighbor analysis. About a month ago, I successfully created a model predicting plant specifications with the help of [Machine Learning with R] (https://www.amazon.com/Machine-Learning-techniques-predictive-modeling/dp/1784393908/ref=sr_1_2_sspa?s=books&ie=UTF8&qid=1519347125&sr=1-2-spons&keywords=machine+learning+in+R&psc=1). Of course, this is probably elementary for many of you here but I still found the process easy to understand and now I'm planning to learn about decision trees and Naive Bayes analysis.

I also did it from a "top down" approach, and then a couple years later I took some MOOCS for some of the math. I'd say the "top down" approach is better. I really enjoyed the math courses, but if your goal is deep learning research, those introductory courses are very broad and will cover a lot of material that's not relevant for you.

I'd first suggest Geometry and Meaning as a great introduction to some of the math of NLP. Note that this isn't focused on deep learning at all (it's from 2004), but it's a great foundation to the math for NLP, and it does a particularly good job of explaining the problems with modeling language, which deep learning helps with. Plus it's short and readable, so you can get through it in a weekend.

I haven't read Bengio's Deep Learning book yet, but it has a section on the relevant math, so that might be good to take a look at as well.

> AutoML. This will be a system for automatically searching and discovering model configurations (algorithm, feature sets, hyper-parameter values, etc.) that result in the best performing models for given modeling problems. The system would also automatically build the production data pipelines to generate the features and labels needed to power the models. We have addressed big pieces of this already with our Feature Store, our unified offline and online data pipelines, and hyper-parameter search feature. We plan to accelerate our earlier data science work through AutoML. The system would allow data scientists to specify a set of labels and an objective function, and then would make the most privacy-and security-aware use of Uber’s data to find the best model for the problem. The goal is to amplify data scientist productivity with smart tools that make their job easier.

Everyone's talking about automating network configurations and hyperparameter tuning, but Uber may be in a favorable situation since they acquired the AI startup Geometric Intelligence last year. Geometric Intelligence had a broad focus on different ML technologies including Artificial Life and their partners included Kenneth Stanley and Joel Lehman. Stanley created the NEAT and HyperNeat algorithms for neuroevolution and the two of them wrote a book about "novelty search" which is a radically different approach to search optimization.

/u/KennethStanley amd /u/joelbot2000 did an AMA a while back: link

Interview with him after he joined Uber: link

Video example of novelty search: link

Count me in!
I really want to read though this book: "https://www.amazon.com/Reinforcement-Learning-Introduction-Adaptive-Computation/dp/0262193981" by Richard Sutton, as well as a few other classical ML books, like Christopher Bishop's and Kevin Murphy's.

I know many concepts already, but I've never studied them in a systematic manner (e.g. follow an 1000-page book from end to end). I hear from multiple friends that it's super beneficial in the long run to build a strong mathematical/statistical foundation.
My current model of "googling here and there" might work in the short term, but will not help me invent new algorithms or improve state-of-the-art.

Matlab works well, but it can be pricey. R, and Python are amazing as a previous comment pointed out. If you're looking for a good IDE for R, I recommend R-studio.

The book we used for my graduate machine learning course was Pattern Classification by Duda, Hart, and Stork. It offers a great overview, and would be quite easy to read for someone with a background in statistics.

This isnt a job posting. I am posting this for a discussion raised from a website I have no connection with.

Firstly these are interesting ideas and seem ideal for blockchain based business models.

Secondly I think the Question at the end about whether these suit men or women is a good one

Thirdly on a weapons of Math destruction level what does it mean to do jobs effecting peoples lives that involve only maths and not meeting the people?

I posted this to start a discussion about the particular ideas and the concept of interaction free jobs and I'd like to hear your opinion

I liked this one Weapons of Math Destruction.

Weapons of Math Destruction and Reality Mining

Read Code Complete

Sutton and Barto wrote the standard text in reinforcement learning.

Here is the AlphaGo paper in Nature.

The FAQ has a list of resources for learning ML including links to Hinton's Coursera course on neural nets.