newtboy said:

It is the greatest country in the world DESPITE it being the home of some of the most disrespectful racist assholes in the world like edit:Thomas E Burtell of Saline Michigan. Sadly, that status is in serious jeopardy because of them. Edit: Fortunately even his own son is disgusted by him and disavowed and condemned his racist and bigoted comments. He identified himself, so I have no qualms about identifying him, but I won't post his address and phone number like others have.

I hope the incessant abuse and ostracism he receives from the community as a whole from this day forward force him to leave not just that school system but this great country...MAGA by getting the fuck out, douchebag.

kir_mokum said:

i'm sure i'm missing something but this seems like a trivial thing to do.

eric3579 said:

If you were like me and had no idea who did this song originally.

newtboy said:

Excellent. I bet that only cost about twice what my car cost. Was anyone else hoping the door would start to open at the end, with lots of steam/smoke and banging?
*promote

This isn't really true though and greatly understates how amazing this demo, and current AI actually is.

Saying the agents are obeying a set of human defined rules / freedoms / constraints and objective functions would lead one to imagine something more like video game AI.

Typically video game AI works on a set of weighted decisions and actions, where the weights, decisions and actions are defined by the developer; a more complex variation of:

if my health is low, move towards the health pack,
otherwise, move towards the opponent

In this demo, no such rules exist. It's not given any weights (health), rules (if health is low), nor any instructions (move towards health pack). I guess you could apply neural networks to traditional game AI to determine the weights for decision making (which are typically hard coded by the developer); but that would be far less interesting than what's actually happening here.

Instead, the agent is given a set of inputs, a set of available outputs, and a goal.

4 Inputs:
- Position of the agent itself
- Position and type (other agent, box, ramp) of objects within a limited forward facing conical view
- Position (but not type) of objects within a small radius around the agent
- Reward: Whether they are doing a good job or not

Note the agent is given no information about each type of object, or what they mean, or how they behave. You may as well call them A, B, C rather than agent, box, ramp.

3 Outputs:
- Move
- Grab
- Lock

Again, the agent knows nothing about what these mean, only that they can enable and disable each at any time. A good analogy is someone giving you a game controller for a game you've never played. The controller has a stick and two buttons and you figure out what they do by using them. It'd be accurate to call the outputs: stick, A, B rather than move, grab, lock.

Goal:
- Do a good job.

The goal is simply for the reward input to be maximised. A good analogy is saying 'good girl' or giving a treat to a dog that you are training when they do the right thing. It's up to the dog to figure out what it is that they're doing that's good.

The reward is entirely separate from the agent, and agent behaviour can be completely changed just by changing when the reward is given. The demo is about hide and seek, where the agents are rewarded for not being seen / seeing their opponent (and not leaving the play area). The agents also succeeded at other games, where the only difference to the agent was when the reward was given.

It isn't really different from physically building the same play space, dropping some rats in it, and rewarding them with cheese when they are hidden from their opponents - except rats are unlikely to figure out how to maximise their reward in such a 'complex' game.

Given this description of how the AI actually works, the fact they came up with complex strategies like blocking doors, ramp surfing, taking the ramp to stop their opponents from ramp surfing, and just the general cooperation with other agents, without any code describing any of those things - is pretty amazing.

You can find out more about how the agents were trained, and other exercises they performed here:

https://openai.com/blog/emergent-tool-use/