simulation implementation idea needed

[DaggyStyle]

hello.
I need to implement a scenario, the scenario composes of 3 groups, two if them with 7 agents each and the last one with 2 agents. each agent contains medium size database.
I thought of 3 ways to implement it:

1. creating a thread for each agent and one of the whole scenario (there is a main thread too), pros: best possible simulation, cons: normal systems (2 cores) will take too long to run it as there can be 18 threads.
2. creating a thread for each group and within the group, each thread runs over the static agents and updates the according to the world in a fixed order. pro: less threads, e.g. game+main+3=5, better to run over normal systems (2 core), cons: losing liveness that makes the simulation less real.
3. using the game thread to do the same like in previous suggestions but over the entire static agents. e.g. 2 threads. pros: excellent performance on normal systems. cons: worst possible simulation.

more things I thought of:

• making the game thread static and reduce the number of threads.
• create a algorithm that according to the number cores I select the group's size
• in cases of 2 and 3, the environment that the agents receive is constant, e.g. if agent a as preformed a action, then agent B that follows see the environment before the action was preformed.

any hints?
thanks in advance.

[durian]

Another variation: a client/server solution, with the different groups running on different computers, connecting to the one running the main thread/program. (Could even run on one computer, but then you'll have the same performance problems).

-peter

[DaggyStyle]

Another variation: a client/server solution, with the different groups running on different computers, connecting to the one running the main thread/program. (Could even run on one computer, but then you'll have the same performance problems).

-peter

for now, this is a single computer software

[timeBandit]

Why does option #2 make your simulation "less real" compared to option #1? Are there non-deterministic effects such that the degree of parallelism is reflected in the simulation results? That would pretty much force your hand to one thread per agent, unless you're able to accept the lost of accuracy.

I think you overestimate the impact of multi-threading. The overhead of a thread context switch is a fraction of that of a process context switch. No matter the architecture you choose, your simulation must complete the work of N agents on X cores, which will likely be the major determinant of execution time. Doing that work with 20 threads vs. 5 will not cripple you on a machine with less than 5 cores.

An option between #1 and #2 is to allocate threads as in option #2, and roll your own cooperative task scheduler to execute the agents within each group thread. Intimate knowledge of your agent routines may allow you to slightly reduce the overhead relative to per-agent threads. Unless this is a big. long-running simulation I doubt it would be worth the effort.

Scaling group size to available cores, listed as your second "more things" bullet, is an excellent idea if it is more important to limit run time than simulate a particular number of agents. In fact, for any chosen approach I'd say include some way to scale your group sizes--you may find it very useful in testing.

[DaggyStyle]

Why does option #2 make your simulation "less real" compared to option #1? Are there non-deterministic effects such that the degree of parallelism is reflected in the simulation results? That would pretty much force your hand to one thread per agent, unless you're able to accept the lost of accuracy.

I think you overestimate the impact of multi-threading. The overhead of a thread context switch is a fraction of that of a process context switch. No matter the architecture you choose, your simulation must complete the work of N agents on X cores, which will likely be the major determinant of execution time. Doing that work with 20 threads vs. 5 will not cripple you on a machine with less than 5 cores.

An option between #1 and #2 is to allocate threads as in option #2, and roll your own cooperative task scheduler to execute the agents within each group thread. Intimate knowledge of your agent routines may allow you to slightly reduce the overhead relative to per-agent threads. Unless this is a big. long-running simulation I doubt it would be worth the effort.

Scaling group size to available cores, listed as your second "more things" bullet, is an excellent idea if it is more important to limit run time than simulate a particular number of agents. In fact, for any chosen approach I'd say include some way to scale your group sizes--you may find it very useful in testing.

each agent looks at the entire world, his own stats, the goal in hand and need to calculate and decide what is the best move to take, I haven't worked out the complete details on how the decision will take place (decisions tree, minimax maybe instant decisions with look ahead depending on the agent's stats). again using thread per agent is best because is it best near reality. but I'm afraid that the decision making will take too much...

I'm not sure I get the difference between #2 and you #1.5 option (the option between #1 and #2)

about the idea, how should I know what is the limit, e.g. for 3 cores, how much threads to allocate? 2 times the cores? what about the intel VT? how should I take that in account?

[durian]

for now, this is a single computer software

You could still run both client and server on the same computer. The advantage would also be a nice separation of the different groups from the main thread, and make it more easily extensible. But that's maybe not required at all :)

-peter