東華大學圖書館 |

Reinforcement Learning for Well Location Optimization.

Record Type:	Electronic resources : Monograph/item
Title/Author:	Reinforcement Learning for Well Location Optimization./
Author:	Dawar, Kshitij.
Published:	Ann Arbor : ProQuest Dissertations & Theses, : 2021,
Description:	234 p.
Notes:	Source: Dissertations Abstracts International, Volume: 83-06, Section: B.
Contained By:	Dissertations Abstracts International83-06B.
Subject:	Petroleum engineering. -
Online resource:	http://pqdd.sinica.edu.tw/twdaoapp/servlet/advanced?query=28929357
ISBN:	9798494497772

Reinforcement Learning for Well Location Optimization.
Dawar, Kshitij.

Reinforcement Learning for Well Location Optimization. - Ann Arbor : ProQuest Dissertations & Theses, 2021 - 234 p.

Source: Dissertations Abstracts International, Volume: 83-06, Section: B.

Thesis (Ph.D.)--The Pennsylvania State University, 2021.

The strategic placement of exploratory wells during the process of hydrocarbon production is critical for both the determination of the reservoir properties and the eventual profitability of the reservoir. Conventional procedures for selecting oil and gas well locations typically collect seismic and other data, conduct extensive simulations and analysis, and then choose the next location for one or more wells. The criteria for selection emphasize maximizing future production. This approach is effectively a single-stage optimization of well location, and it ignores the value of information from initial well locations in reducing the uncertainty about the subsurface properties of the reservoir. In addition to the extraction of hydrocarbon resources, wells provide insights regarding the petrophysical and geological properties of the reservoir and are the primary source of information to update reservoir models leading to the better characterization of the reservoir. In the initial stages of reservoir development, the spatial characterization of geological properties have significant uncertainty because of the scarcity of direct subsurface measurements and the poor granularity of auxiliary information such as seismic data. Each new well location, in addition to its production, provides information that can reduce the uncertainty about reservoir properties at other locations and improve the selection of subsequent well locations. In essence, the placement of initial wells entails a tradeoff between exploration, enabling better subsequent well locations, and exploitation, maximizing profitability of the next well location. In general, single-stage optimization approaches that optimize each decision sequentially often do not lead to the maximum cumulative benefit. A better approach is to select well locations within a multi-stage framework that accounts for the information value from well placement on subsequent reservoir development decisions.In this dissertation, I frame the well location selection problem as a multi-stage sequential decision under uncertainty. My contribution is to apply reinforcement learning techniques and to formulate sequential well location selection as a dynamic programming problem. Within the reinforcement learning framework, I apply geostatistical simulation techniques to characterize the uncertainty in reservoir models and to determine the updates to the beliefs regarding reservoir rock properties that would result from alternative well location decisions and hypothetical observations at those locations. Because of the number of feasible sequences of well locations, the possible observations within each sequence, and the computational demands of simulating updated reservoir properties, traditional dynamic programming solution methods are not tractable. I demonstrate the application of reinforcement learning, or approximate dynamic programming, a class of algorithms that combine Monte Carlo sampling methods with functional approximations of the objective function. These methods provide a computationally tractable approach to exploring sequential well location selection with explicit consideration of the information value of initial well locations.Using this approach, I test the hypothesis that a well location selection strategy is more robust to uncertainty in the initial data than single-stage optimization approaches. To test this hypothesis, I apply a novel reinforcement learning framework to select well locations accounting for information value. I develop several proxy geostatistical models to reduce the computational time and effort and explore the effects of the choice of proxy model on the suggested well placement strategy. I demonstrate the solution of the well location problem using one reinforcement learning algorithm, Q-learning, and discuss its limitations. I first demonstrate the framework on a simpler two-dimensional well location problem, using both tabular approaches and artificial neural networks as functional approximators. In the 2D case, I explore the sensitivity of the policy developed using reinforcement learning to the hyperparameters that control the exploration of the state-action space, and I demonstrate convergence to the optimal policy. I then apply the deep reinforcement learning algorithm to the Stanford V and SPE comparative solutions project model 2 reservoirs, which are larger three-dimensional reservoir cases. The results of the larger reservoir cases demonstrate the benefits of sequential well location selection relative to single-stage optimization approaches.

ISBN: 9798494497772Subjects--Topical Terms:

566616
Petroleum engineering.
Subjects--Index Terms:

Well location selection

Reinforcement Learning for Well Location Optimization.
LDR:05878nmm a2200421 4500 001 2349657
005 20230509091140.5
006 m o d
007 cr#unu||||||||
008 241004s2021 ||||||||||||||||| ||eng d
020 $a 9798494497772
035 $a (MiAaPQ)AAI28929357
035 $a AAI28929357
040 $a MiAaPQ $c MiAaPQ
100 1 $a Dawar, Kshitij. $3 3689072
245 1 0 $a Reinforcement Learning for Well Location Optimization.
260 1 $a Ann Arbor : $b ProQuest Dissertations & Theses, $c 2021
300 $a 234 p.
500 $a Source: Dissertations Abstracts International, Volume: 83-06, Section: B.
500 $a Advisor: Srinivasan, Sanjay;Webster, Mort D.
502 $a Thesis (Ph.D.)--The Pennsylvania State University, 2021.
520 $a The strategic placement of exploratory wells during the process of hydrocarbon production is critical for both the determination of the reservoir properties and the eventual profitability of the reservoir. Conventional procedures for selecting oil and gas well locations typically collect seismic and other data, conduct extensive simulations and analysis, and then choose the next location for one or more wells. The criteria for selection emphasize maximizing future production. This approach is effectively a single-stage optimization of well location, and it ignores the value of information from initial well locations in reducing the uncertainty about the subsurface properties of the reservoir. In addition to the extraction of hydrocarbon resources, wells provide insights regarding the petrophysical and geological properties of the reservoir and are the primary source of information to update reservoir models leading to the better characterization of the reservoir. In the initial stages of reservoir development, the spatial characterization of geological properties have significant uncertainty because of the scarcity of direct subsurface measurements and the poor granularity of auxiliary information such as seismic data. Each new well location, in addition to its production, provides information that can reduce the uncertainty about reservoir properties at other locations and improve the selection of subsequent well locations. In essence, the placement of initial wells entails a tradeoff between exploration, enabling better subsequent well locations, and exploitation, maximizing profitability of the next well location. In general, single-stage optimization approaches that optimize each decision sequentially often do not lead to the maximum cumulative benefit. A better approach is to select well locations within a multi-stage framework that accounts for the information value from well placement on subsequent reservoir development decisions.In this dissertation, I frame the well location selection problem as a multi-stage sequential decision under uncertainty. My contribution is to apply reinforcement learning techniques and to formulate sequential well location selection as a dynamic programming problem. Within the reinforcement learning framework, I apply geostatistical simulation techniques to characterize the uncertainty in reservoir models and to determine the updates to the beliefs regarding reservoir rock properties that would result from alternative well location decisions and hypothetical observations at those locations. Because of the number of feasible sequences of well locations, the possible observations within each sequence, and the computational demands of simulating updated reservoir properties, traditional dynamic programming solution methods are not tractable. I demonstrate the application of reinforcement learning, or approximate dynamic programming, a class of algorithms that combine Monte Carlo sampling methods with functional approximations of the objective function. These methods provide a computationally tractable approach to exploring sequential well location selection with explicit consideration of the information value of initial well locations.Using this approach, I test the hypothesis that a well location selection strategy is more robust to uncertainty in the initial data than single-stage optimization approaches. To test this hypothesis, I apply a novel reinforcement learning framework to select well locations accounting for information value. I develop several proxy geostatistical models to reduce the computational time and effort and explore the effects of the choice of proxy model on the suggested well placement strategy. I demonstrate the solution of the well location problem using one reinforcement learning algorithm, Q-learning, and discuss its limitations. I first demonstrate the framework on a simpler two-dimensional well location problem, using both tabular approaches and artificial neural networks as functional approximators. In the 2D case, I explore the sensitivity of the policy developed using reinforcement learning to the hyperparameters that control the exploration of the state-action space, and I demonstrate convergence to the optimal policy. I then apply the deep reinforcement learning algorithm to the Stanford V and SPE comparative solutions project model 2 reservoirs, which are larger three-dimensional reservoir cases. The results of the larger reservoir cases demonstrate the benefits of sequential well location selection relative to single-stage optimization approaches.
590 $a School code: 0176.
650 4 $a Petroleum engineering. $3 566616
650 4 $a Industrial engineering. $3 526216
650 4 $a Statistics. $3 517247
650 4 $a Petroleum geology. $3 3173827
650 4 $a Operations research. $3 547123
653 $a Well location selection
653 $a Multi-stage sequential decision
653 $a Uncertainty
653 $a Reinforcement learning
653 $a Dynamic programming
653 $a Sequential well location selection
690 $a 0765
690 $a 0546
690 $a 0583
690 $a 0463
690 $a 0796
710 2 $a The Pennsylvania State University. $3 699896
773 0 $t Dissertations Abstracts International $g 83-06B.
790 $a 0176
791 $a Ph.D.
792 $a 2021
793 $a English
856 4 0 $u http://pqdd.sinica.edu.tw/twdaoapp/servlet/advanced?query=28929357