# CSE675 References

### 1. Simulation of Markov Processes

#### 1.1 What is simulation modeling and why should we care? Review of probability theory.

• Stochastic modelling for systems biology [1], Chapter 1: What is simulation modeling? Why simulation modeling in systems biology?
• The Big Book of Simulation Modeling: Multimethod Modeling with AnyLogic 6 [2], Chapter 1: What is simulation modeling? Why simulation modeling in studying complex social systems?
• Generative Social Science: Studies in Agent-Based Computational Modeling (Princeton Studies in Complexity) [3], Chapter 1, The Generativist Manifesto, we do not understand “emergence” if we cannot construct it using agent-based modeling.
• Interaction process analysis; a method for the study of small groups [4]: Social psychology, while seemingly too complicated to be modeled, has models

#### 1.2 Where are the probability distributions (Bernoulli trial, binomial, multinomial, order statistics, exponential, Poisson, Gaussian and more) from?

• Stochastic Simulation [5]: sampling techniques
• Simulation Modeling and Analysis [6]: sampling techniques
• Probabilities, Statistics and Data Modeling: An interesting e-book on where different probability distributions are originated, estimation and statistical testing, published by AI ACCESS (aiaccess@aol.com). Unfortunately this book is no longer available. I will share chapters later.

#### 1.3 What are the random processes we often encounter? How do we sample from them?

• Stochastic modelling for systems biology [1], chapter 6 on relating microscopic molecule-level interactions and macroscopic dynamics, chapter 8 on approximate algorithms.
• Financial Modeling with Markov Jump Processes [7], on simulating Markov jump processes.

### 2. Making Inferences with Markov Processes

#### 2.1 Exact inference with message passing and expectation maximization with applications to hidden Markov models and Kalman filter

• A new approach to linear filtering and prediction problems [8], this is where Kalman filter was first brought forth by Kalman
• Kalman filter on Wikipedia.
• Extended Kalman filter is first order approximation to the state transition kernel when it is non-linear. Unscented Kalman filter is reported to out-perform extended Kalman filter.
• Hidden Markov model: formulation, forward-backward method for latent state inference, EM algorithm for parameter estimation, Viterbi decoding algorithm to find maximum likelihood latent state path.

#### 2.2 Legendre transform, K-L divergence, with applications to Markov processes that defy exact inference

• Graphical Models, Exponential Families, and Variational Inference [9]: the best theoretical treatment of this topic, with loop belief propagation and expectation propagation discussed in chapter 4, mean field method in chapter 5 and parameter learning in chapter 6.
• Factorial Hidden Markov Models [10]: This is  where Ghahramani and Jordan first applied mean field method to make inferences with graphical models.
• Expectation Propogation for Approximate Inference in Dynamic Bayesian Networks [11]: duality between minimizing Bethe variation and minimizing negative log likelihood.
• Expectation Propagation for approximate Bayesian inference [12]: This is where EP was first brought forth by Minka.

#### 2.3 Markov chain Monte Carlo, with applications to Markov processes that defy exact inference

• Stochastic modelling for systems biology [1]: chapter 9 on Gibbs sampling and Metropolis-Hastings sampling, chapter 10 on sampling based inference of stochastic kinetic models.

### 3. Complex Systems identified by Markov Processes

#### 3.1 Evolution of social networks: random graph model, exponential random graph model, network formation game

• Exponential Random Graph Models for Social Networks [13]: ERGM simulation, inference, parameter learning, applications; Temporal ERGM.
• Game theory models of network formation: A strategic model of social and economic networks [14];The evolution of social and economic networks [15].
• statnet, a suite of software packages for network analysis that implement recent advances in the statistical modeling of networks.
• Physicists models on network evolution: On Random Graphs [16], On the Evolution of Random Graphs [17], Collective dynamics of ‘small-world’networks [18], Small worlds: the dynamics of networks between order and randomness [19], Emergence of scaling in random networks [20].
• Real world network formation: Reality Commons data set and a network tracked with personal mobile phones [21].

#### 3.2 Diffusion in social networks: opinion dynamics, culture dynamics & language dynamics

• Statistical Physics of Social Dynamics [22]: Opinion dynamics, formation of languages and cultures, etc.
• Interacting Particle Systems [23].
• Agent-Based Models (Quantitative Applications in Social Sciences) [24]
• NetLogo
• Diffusion in real world: The spread of obesity in a large social network over 32 years [25] …

#### 3.3 Urban dynamics: formation of cities, road transportation dynamics

• Cities and complexity: understanding cities with cellular automata, agent-based models, and fractals [26]
• Urban Dynamics [27]: book on the interaction between different industrial sections using system dynamics approach.
• TRANSIMS: Transportation analysis and simulation system [28]
• MATSIM-T: Aims, approach and implementation [29]

### References

[1] D. J. Wilkinson, Stochastic modelling for systems biology, Boca Raton, FL: Taylor & Francis, 2006.
[Bibtex]
@Book{Wilkinson_Stochastic_modelling_2006,
Title = {Stochastic modelling for systems biology},
Author = {Darren James. Wilkinson},
Publisher = {Taylor \& Francis},
Year = {2006},
Date-modified = {2012-08-11 18:32:17 +0000}
}
[2] A. Borshchev, The big book of simulation modeling: multimethod modeling with anylogic 6, AnyLogic North America, 2013.
[Bibtex]
@Book{borshchev2013big,
Title = {The Big Book of Simulation Modeling: Multimethod Modeling with AnyLogic 6},
Author = {Borshchev, Andrei},
Publisher = {AnyLogic North America},
Year = {2013},
Date-modified = {2015-09-14 04:04:43 +0000}
}
[3] J. M. M. Epstein, Generative social science: studies in agent-based computational modeling (princeton studies in complexity), Princeton University Press, 2007.
[Bibtex]
@Book{Epstein07GenerativeSocialScience,
Title = {Generative Social Science: Studies in Agent-Based Computational Modeling (Princeton Studies in Complexity)},
Author = {Joshua M. M. Epstein},
Publisher = {Princeton University Press},
Year = {2007},
Date-modified = {2012-07-23 15:05:32 +0000}
}
[4] R. F. Bales, “Interaction process analysis; a method for the study of small groups.,” , 1950.
[Bibtex]
@Article{bales1950interaction,
Title = {Interaction process analysis; a method for the study of small groups.},
Author = {Bales, Robert F},
Year = {1950},
}
[5] B. D. Ripley, Stochastic simulation, John Wiley & Sons, 2009, vol. 316.
[Bibtex]
@Book{ripley2009stochastic,
Title = {Stochastic simulation},
Author = {Ripley, Brian D},
Publisher = {John Wiley \& Sons},
Year = {2009},
Volume = {316}
}
[6] D. W. Kelton and A. M. Law, Simulation modeling and analysis, McGraw Hill Boston, 2000.
[Bibtex]
@Book{kelton2000simulation,
Title = {Simulation modeling and analysis},
Author = {Kelton, W David and Law, Averill M},
Publisher = {McGraw Hill Boston},
Year = {2000}
}
[7] P. Tankov, Financial modelling with jump processes, Boca Raton, Fla.: Chapman & Hall/CRC, 2004.
[Bibtex]
@Book{Cont_Financial_modelling_c2004,
Title = {Financial modelling with jump processes},
Author = {Peter Tankov},
Publisher = {Chapman \& Hall/CRC},
Year = {2004},
Date-modified = {2012-08-11 18:32:17 +0000}
}
[8] R. E. Kalman, “A new approach to linear filtering and prediction problems,” Journal of fluids engineering, vol. 82, iss. 1, pp. 35-45, 1960.
[Bibtex]
@Article{kalman1960new,
Title = {A new approach to linear filtering and prediction problems},
Author = {Kalman, Rudolph Emil},
Journal = {Journal of Fluids Engineering},
Year = {1960},
Number = {1},
Pages = {35--45},
Volume = {82},
Publisher = {American Society of Mechanical Engineers}
}
[9] M. J. Wainwright and M. I. Jordan, “Graphical models, exponential families, and variational inference,” Foundations and trends in machine learning, vol. 1, iss. 1-2, pp. 1-305, 2008.
[Bibtex]
@Article{DBLP:journals/ftml/WainwrightJ08,
Title = {Graphical Models, Exponential Families, and Variational Inference},
Author = {Martin J. Wainwright and Michael I. Jordan},
Journal = {Foundations and Trends in Machine Learning},
Year = {2008},
Number = {1-2},
Pages = {1-305},
Volume = {1},
Bibsource = {DBLP, http://dblp.uni-trier.de},
Date-modified = {2012-08-11 18:32:17 +0000},
Ee = {http://dx.doi.org/10.1561/2200000001}
}
[10] Z. Ghahramani and M. I. Jordan, “Factorial hidden markov models,” Machine learning, vol. 29, iss. 2-3, pp. 245-273, 1997.
[Bibtex]
@Article{DBLP:journals/ml/GhahramaniJ97,
Title = {Factorial Hidden Markov Models},
Author = {Zoubin Ghahramani and Michael I. Jordan},
Journal = {Machine Learning},
Year = {1997},
Number = {2-3},
Pages = {245-273},
Volume = {29},
Bibsource = {DBLP, http://dblp.uni-trier.de},
Date-modified = {2012-08-11 18:32:17 +0000},
Ee = {http://dx.doi.org/10.1023/A:1007425814087}
}
[11] T. Heskes and O. Zoeter, “Expectation propogation for approximate inference in dynamic bayesian networks,” in Uai, 2002, pp. 216-223.
[Bibtex]
@InProceedings{DBLP:conf/uai/HeskesZ02,
Title = {Expectation Propogation for Approximate Inference in Dynamic Bayesian Networks},
Author = {Tom Heskes and Onno Zoeter},
Booktitle = {UAI},
Year = {2002},
Pages = {216-223},
Bibsource = {DBLP, http://dblp.uni-trier.de},
Crossref = {DBLP:conf/uai/2002},
Date-modified = {2012-10-06 08:56:47 +0000},
Ee = {http://uai.sis.pitt.edu/displayArticleDetails.jsp?mmnu=1{\&}smnu=2{\&}article_id=863{\&}proceeding_id=18}
}
[12] T. P. Minka, “Expectation propagation for approximate bayesian inference,” in Uai, 2001, pp. 362-369.
[Bibtex]
@InProceedings{DBLP:conf/uai/Minka01,
Title = {Expectation Propagation for approximate Bayesian inference},
Author = {Thomas P. Minka},
Booktitle = {UAI},
Year = {2001},
Pages = {362-369},
Bibsource = {DBLP, http://dblp.uni-trier.de},
Crossref = {DBLP:conf/uai/2001},
Date-modified = {2012-10-06 08:54:23 +0000},
Ee = {http://uai.sis.pitt.edu/displayArticleDetails.jsp?mmnu=1{\&}smnu=2{\&}article_id=120{\&}proceeding_id=17}
}
[13] G. Robins, “Exponential random graph models for social networks,” Handbook of social network analysis. sage, 2011.
[Bibtex]
@article{robins2011exponential,
title={Exponential random graph models for social networks},
author={Robins, Garry},
journal={Handbook of Social Network Analysis. Sage},
year={2011},
publisher={Citeseer}
}
[14] M. O. Jackson and A. Wolinsky, “A strategic model of social and economic networks,” Journal of economic theory, vol. 71, iss. 1, pp. 44-74, 1996.
[Bibtex]
@article{jackson1996strategic,
title={A strategic model of social and economic networks},
author={Jackson, Matthew O and Wolinsky, Asher},
journal={Journal of economic theory},
volume={71},
number={1},
pages={44--74},
year={1996},
publisher={Elsevier}
}
[15] M. O. Jackson and A. Watts, “The evolution of social and economic networks,” Journal of economic theory, vol. 106, iss. 2, pp. 265-295, 2002.
[Bibtex]
@article{jackson2002evolution,
title={The evolution of social and economic networks},
author={Jackson, Matthew O and Watts, Alison},
journal={Journal of Economic Theory},
volume={106},
number={2},
pages={265--295},
year={2002},
publisher={Elsevier}
}
[16] P. ERDdS and A. R&WI, “On random graphs i,” Publ. math. debrecen, vol. 6, pp. 290-297, 1959.
[Bibtex]
@article{erdds1959random,
title={On random graphs I},
author={ERDdS, P and R\&WI, A},
journal={Publ. Math. Debrecen},
volume={6},
pages={290--297},
year={1959}
}
[17] P. Erd6s and A. Rényi, “On the evolution of random graphs,” Publ. math. inst. hungar. acad. sci, vol. 5, pp. 17-61, 1960.
[Bibtex]
@article{erd6s1960evolution,
title={On the evolution of random graphs},
author={Erd6s, Paul and R{\'e}nyi, A},
journal={Publ. Math. Inst. Hungar. Acad. Sci},
volume={5},
pages={17--61},
year={1960},
publisher={Citeseer}
}
[18] D. J. Watts and S. H. Strogatz, “Collective dynamics of ‘small-world’networks,” Nature, vol. 393, iss. 6684, pp. 440-442, 1998.
[Bibtex]
@article{watts1998collective,
title={Collective dynamics of ‘small-world’networks},
author={Watts, Duncan J and Strogatz, Steven H},
journal={nature},
volume={393},
number={6684},
pages={440--442},
year={1998},
publisher={Nature Publishing Group}
}
[19] D. J. Watts, Small worlds: the dynamics of networks between order and randomness, Princeton university press, 1999.
[Bibtex]
@book{watts1999small,
title={Small worlds: the dynamics of networks between order and randomness},
author={Watts, Duncan J},
year={1999},
publisher={Princeton university press}
}
[20] A. Barabási and R. Albert, “Emergence of scaling in random networks,” Science, vol. 286, iss. 5439, pp. 509-512, 1999.
[Bibtex]
@article{barabasi1999emergence,
title={Emergence of scaling in random networks},
author={Barab{\'a}si, Albert-L{\'a}szl{\'o} and Albert, R{\'e}ka},
journal={science},
volume={286},
number={5439},
pages={509--512},
year={1999},
publisher={American Association for the Advancement of Science}
}
[21] W. Dong, B. Lepri, and A. Pentland, “Modeling the co-evolution of behaviors and social relationships using mobile phone data,” in Mum, 2011, pp. 134-143.
[Bibtex]
@InProceedings{DBLP:conf/mum/DongLP11,
Title = {Modeling the co-evolution of behaviors and social relationships using mobile phone data},
Author = {Wen Dong and Bruno Lepri and Alex Pentland},
Booktitle = {MUM},
Year = {2011},
Pages = {134-143},
Bibsource = {DBLP, http://dblp.uni-trier.de},
Crossref = {DBLP:conf/mum/2011},
Date-modified = {2012-08-11 18:32:17 +0000},
Ee = {http://doi.acm.org/10.1145/2107596.2107613}
}
[22] C. Castellano, S. Fortunato, and V. Loreto, “Statistical physics of social dynamics,” Reviews of modern physics, vol. 81, iss. 2, pp. 591-646, 2009.
[Bibtex]
@Article{Castellano09SocialDynamics,
Title = {Statistical physics of social dynamics},
Author = {Claudio Castellano and Santo Fortunato and Vittorio Loreto},
Journal = {Reviews of Modern Physics},
Year = {2009},
Number = {2},
Pages = {591-646},
Volume = {81},
Date-modified = {2012-07-30 12:24:17 +0000}
}
[23] T. Liggett, Interacting particle systems, Springer Science & Business Media, 2012, vol. 276.
[Bibtex]
@book{liggett2012interacting,
title={Interacting particle systems},
author={Liggett, Thomas},
volume={276},
year={2012},
}
[24] N. Gilbert, Agent-based models (quantitative applications in social sciences), Sage Publications, Inc., 2007.
[Bibtex]
@Book{Nigel07ABM,
Title = {Agent-Based Models (Quantitative Applications in Social Sciences)},
Author = {Nigel Gilbert},
Publisher = {Sage Publications, Inc.},
Year = {2007},
Date-modified = {2012-07-30 15:33:34 +0000}
}
[25] N. A. Christakis and J. H. Fowler, “The spread of obesity in a large social network over 32 years,” New england journal of medicine, vol. 357, iss. 4, pp. 370-379, 2007.
[Bibtex]
@article{christakis2007spread,
title={The spread of obesity in a large social network over 32 years},
author={Christakis, Nicholas A and Fowler, James H},
journal={New England journal of medicine},
volume={357},
number={4},
pages={370--379},
year={2007},
publisher={Mass Medical Soc}
}
[26] M. Batty, Cities and complexity: understanding cities with cellular automata, agent-based models, and fractals, Mit Press, 2007.
[Bibtex]
@Book{batty2007cities,
Title = {Cities and Complexity: Understanding Cities With Cellular Automata, Agent-Based Models, and Fractals},
Author = {Batty, M.},
Publisher = {Mit Press},
Year = {2007},
Date-modified = {2012-07-24 01:21:51 +0000},
ISBN = {9780262524797},
}
[27] J. W. Forrester, Urban dynamics, Massachusetts Institute of Technology, 1970, vol. 11.
[Bibtex]
@Book{forrester1970urban,
Title = {Urban dynamics},
Author = {Forrester, Jay W},
Publisher = {Massachusetts Institute of Technology},
Year = {1970},
Number = {3},
Volume = {11},
Date-modified = {2015-10-08 14:53:42 +0000},
Journal = {IMR; Industrial Management Review (pre-1986)},
Keywords = {modeling},
Pages = {67}
}
[28] L. Smith, R. Beckman, and K. Baggerly, “Transims: transportation analysis and simulation system,” Los Alamos National Lab., NM (United States) 1995.
[Bibtex]
@TechReport{smith1995transims,
Title = {TRANSIMS: Transportation analysis and simulation system},
Author = {Smith, Laron and Beckman, Richard and Baggerly, Keith},
Institution = {Los Alamos National Lab., NM (United States)},
Year = {1995},
}
@TechReport{matsim,
}