Algorithms for Smart World Technologies

Chapter 9Decision-making Algorithms

In complex environments such as energy, environment, and resource management, decision-making processes often involve uncertainty, multiple objectives, and dynamically changing conditions. This chapter introduces algorithmic frameworks and models designed to aid decision-making in such contexts. Beginning with the theoretical foundation of Markov decision processes (MDPs), it extends to practical methods like reinforcement learning (RL), value iteration, Q-learning, and temporal difference (TD) learning. These algorithms are particularly valuable in autonomous systems, environmental monitoring, and adaptive energy networks, where agents must learn optimal strategies through interaction with the environment. By the end of this chapter, readers will understand how decision-making models operate, how policies are optimized over time, and how to apply these techniques in real-world scenarios. After reading this chapter, you should be able to:

• Explain the fundamental components of decision-making frameworks such as MDPs and RL.
• Identify appropriate decision models for environmental and energy infrastructure systems.
• Distinguish between deterministic and stochastic policies, and between model-based and model-free learning approaches.
• Apply algorithms such as value iteration, Q-learning, and TD learning to solve decision-making problems under uncertainty.
• Interpret the role of reward functions, discount factors, and policy convergence in long-term ...