score: 483 Expectimax Pacman Won 1/5 Avg. follow the random policy, which chooses a legal move uniformly at random. While I was tweaking the coefficients, I thought back to Andrew Ng’s machine learning course on coursera. tries to eat as many food pellets (the small white dots) as possible, while Again, the minimax values of Pacman, now with ghosts. I expected gradient descent to win out over my hand-made algorithm. Multi Agent Games for Pacman In this post I want to show compact, simple and elegant way of implementing agents for Pacman Game using python. avoiding the ghosts (the other two agents with eyes in the above figure). I only stuck with it for a few weeks, but that was enough to give me the intuition that this problem would lend itself to linear regression/batch gradient descent. write it in terms of the same functions that were specified in problem 1a. the rest are ghosts. Running the command given above this just a few seconds per move or faster. A short paragraph answering the questions above. Pac-Man acts as a max agent, and the ghosts act as min Minimax, Expectimax, Evaluation Introduction In this project, you will design agents for the classic version of Pacman, including ghosts. This generated a comma-separated data file, where the first six entries are generated coefficients, and the last entry is the average score for pacman over the course of 10 games. For me, this means that I continued to work on the assignments even after I had passed the threshold for getting what I considered a good grade. In expectimax search, we have a probabilistic model of how the opponent (or environment) will behave in any state Model could be a simple uniform distribution (roll a die) Model could be sophisticated and require a great deal of computation $\text{Player}(s)$, which returns the player whose turn it is in state $s$. The last value in each line is the average score over 10 games withthat set of coefficients. All states in minimax should be GameStates, either passed in to getAction or generated via GameState.generateSuccessor. It turns out that the pacman system allows the user to send an arbitrary number of arguments to a pac-man agent. Multi-Agent Search. -492 for depths 1, 2, 3, and 4, respectively. an agent’s belief that it’s raining, given the temperature  E.g. bottom-right corner are capsules, which give Pac-Man power to eat ghosts in a the initial state in the minimaxClassic layout are 9, 8, 7, and Project 1: Multi-Agent Pac-Man. We are giving an exhaustive and very detailed description below, for the sake of completeness and to save you from digging deeper into the starter code. In this project, you will not be abstracting to simplified states. Minimax, Expectimax, Evaluation. Minimax, Expectimax, Evaluation Introduction. Expectimax for Pacman 22 Pacman used depth 4 search with an evalfunction that avoids trouble Ghost used depth 2 search with an evalfunction that seeks Pacman Minimizing Ghost Random Ghost Minimax Pacman Won 5/5 Avg. You may want to use the reciprocal of important values rather than the values themselves for your features. $\text{Succ}(s,a)$, which returns the successor state resulting from taking an action $a$ at a certain state $s$. pruning to more efficiently explore the minimax tree in. Please write your thoughts in. This evaluation function is meant for use with adversarial search agents (not reflex agents). """ For reference, my assignment instructions explained: With depth 2 search, your evaluation function should clear the smallClassic layout with two random ghosts more than half the time and still run at a reasonable rate (to get full credit, Pacman should be averaging around 1000 points when he’s winning). that agent is flipping any coins! Score: 483 Won 5/5 Avg. In this project, you will design agents for the classic version of Pac-Man, including ghosts. I chose an alpha value of .01, and my algorithm had no trouble finding a minimum. self.evaluationFunction. python pacman.py -p ExpectimaxAgent -l trappedClassic -a depth=3 -q -n 10. Minimax, Expectimax, Evaluation. Pacman hovered around a 80-90% win rate, and was able to consistently average over 1100 points, including the points he got when he died. My manual tweaks worked really well. In this project, you will design agents for the classic version of Pac-Man, including ghosts. Question 4 (25 points) Expectimax Minimax and alpha-beta are great, but they both assume that you are playing against an adversary who makes optimal decisions. The score is the same one displayed in the Pacman GUI. In this assignment, you will design agents for the classic version of Pac-Man, including ghosts. I implemented breadth first, depth first, uniform cost, and A* search algorithms to help Pacman find dots and eat them efficiently.. With enemies added to the mix, I implemented minimax and expectimax searches to help Pacman eat dots while avoiding his ghost adversaries. "opt1=val1,opt2,opt3=val3"', Andrew Ng’s machine learning course on coursera. On Gradescope, your total assignment test points (for questions 1, 2 & 3) add up to 30, but the max points will show 40. limited time window, but you won't be worrying about them for the required MinimaxAgent minimax values, although the actions it selects can Students implement multiagent minimax and expectimax algorithms, as well as designing evaluation functions. vary because of different tie-breaking behavior. makes it seem like we’ve found the lower limit for the cost. The AI course I’m enrolled in at Northeastern, CS4100/5100, is currently using the pac-man projects, and I think it’s a great idea. Code for reading layout files and storing their contents. An Expectimax agent no longer makes the assumption that it is playing Problem 4: Evaluation function (extra credit). self.evaluationFunction, which defaults to populated from the command line options. They’re taken in as comma-separated arguments to pacman.py itself, after the -a flag, after specifying the agent. This evaluation function is meant for use with adversarial search agents (not reflex agents). """ Options: Default ghosts are random; you can also play for fun with slightly smarter directional ghosts using -g DirectionalGhost. which is the maximum expected utility against ghosts that each I moved this script and the pacman files to my cloud server, let it run for a day or so, and picked up about 2,234 training samples. As a student, I think it’s often too easy to get caught up in treating your classes and assignments as adversaries, working for hours just to get something done, or just to get a grade, rather than working for your own personal and professional development. assignments, you will only be modifying submission.py. The implementation of minimax and expectimax search is beyond the scope of this post, but it’s important to know that for those search to work properly, it needs to be able to accurately measure the value of game states. You should now observe a more cavalier approach to close quarters with Question 3: Expectimax Agent (worth 20/100 marks) You will now implement an Expectimax agent to play the game of Pacman. Minimax, Expectimax. paragraph, which uses the default mediumClassic layout, Along the way, you will implement both minimax and expectimax search and try your hand at evaluation function design. Random ghosts are of course not optimal minimax agents, and so modeling them with minimax search may not be appropriate. After modfiying the MultiAgents.py MultiAgentSearchAgent constructor to take in the coefficients as parameters, I was able to store them as global variables, and then access them in my evaluation function. In this project, you will design agents for the classic version of Pacman, including ghosts. Before you code up Pac-Man This involved designing state evaluation functions. As in previous $V_{\text{minmax}}(s,d)$ in math as a. Your assignment is graded out of 30. A pacman state contains information like the positions of all the ghosts in the maze, the list of all the food (dots), etc. the ghosts)? In this question you will implement the ExpectimaxAgent, which is useful for modeling probabilistic behavior of agents who may make suboptimal choices. This contest involves a multiplayer capture-the-flag variant of Pacman, where agents control both Pacman … How can you add multiple features to your evaluation function? The second assignment included implementing minimax, expectimax, alpha-beta pruning, and designing an evaluator function to estimate the utility of states. Computational social science researcher living in Boston, Comparing the hand-made function to gradient descent, 'Comma seperated values sent to agent. Pac-Man is always agent 0, and the agents move in order of increasing agent index. eats all the food in a maze, it wins. Make sure your minimax code makes Expectimax Pseudocode def value(s) if s is a max node return maxValue(s) if s is an exp node return expValue(s) ... Expectimax for Pacman • Notice that we’ve gotten away from thinking that the ghosts are trying to minimize pacman’s score All states in minimax should be GameStates, either passed in to getAction or generated via GameState.generateSuccessor. One layer of "depth" includes pacman and all of the opponent agents. All states in minimax should be GameStates, either passed in to getActionor generated via GameState.generateSuccessor. These are present only to Introduction. I then wrote up a short script to generate the training samples, by calling the pacman game with generated coefficients. minimax agents, so modeling them with minimax search is not Your recurrence should resemble that of problem 1a, which means that you should Instead, write down the recurrence for $V_{\text{exptmax}}(s,d)$, return currentGameState. Q4 The expectimax agent survived about half the time as expected. You should see a speed-up: Perhaps depth 3 alpha-beta will run as fast as depth 2 search will involve Pac-Man and each ghost moving two times. Why does Pac-Man's behavior as an expectimax agent differ from his behavior score: 503 Results from playing 5 games While Minimax assumes that the adversary (the minimizer) plays optimally, the Expectimax doesn’t. Write the recurrence for As anyone who has ever won tic-tac-toe can tell you, this is not always the case. Score: 493 Won 5/5 Avg. Pac-Man is always agent 0, and the agents move in order of increasing agent index. The big white dots at the top-left and Pac-Man is always agent 0, and the agents move in order of increasing agent index. Along the way, you will implement both minimax and expectimax search and try your hand at evaluation function design. Pac-Man, now with ghosts. particular. I chose to compute the utility of some state as a linear combination of a series of features: You can see the full function, along with an explanation of the coefficients I used, here. getScore () Students implement multiagent minimax and expectimax algorithms, as well as designing evaluation functions. That’s pretty big. are $n+1$ agents on the board, $a_0,\ldots , a_n$, where $a_0$ is Pac-Man and include in the write-up. Project 2: Multi-Agent Search Pacman, now with ghosts. score:-303 Won 5/5 Avg. one adversary, Pac-Man could have multiple ghosts as adversaries. Pac-Man, now with ghosts. Clearly describe your evaluation function. taught in class. Pacman Won 5/5 Avg.Score: 483 Won 5/5 Avg.Score: 493 Expectimax Pacman Won 1/5 Avg.Score: -303 Won 5/5 Avg.Score: 503 [Demos: world assumptions (L7D3,4,5,6)] Results from playing 5 games Pacmanused depth 4 search with an evalfunction that avoids trouble Ghost used depth 2 search with an evalfunction that seeks Pacman