PetscIJacobian.cpp File Reference

Contains the functions required for Jacobian computations for implicit time-integration. More...

#include <stdlib.h>
#include <basic.h>
#include <arrayfunctions.h>
#include <mpivars_cpp.h>
#include <simulation_object.h>
#include <petscinterface.h>

Go to the source code of this file.

Macros
#define	__FUNCT__   "PetscIJacobian"
#define	__FUNCT__   "PetscJacobianFunction_JFNK"
#define	__FUNCT__   "PetscJacobianFunction_Linear"

Functions
PetscErrorCode	PetscIJacobian (TS ts, PetscReal t, Vec Y, Vec Ydot, PetscReal a, Mat A, Mat B, void *ctxt)
PetscErrorCode	PetscJacobianFunction_JFNK (Mat Jacobian, Vec Y, Vec F)
PetscErrorCode	PetscJacobianFunction_Linear (Mat Jacobian, Vec Y, Vec F)

Detailed Description

Contains the functions required for Jacobian computations for implicit time-integration.

Author

Debojyoti Ghosh

Definition in file PetscIJacobian.cpp.

Macro Definition Documentation

#define __FUNCT__   "PetscIJacobian"

Definition at line 188 of file PetscIJacobian.cpp.

#define __FUNCT__   "PetscJacobianFunction_JFNK"

Definition at line 188 of file PetscIJacobian.cpp.

#define __FUNCT__   "PetscJacobianFunction_Linear"

Definition at line 188 of file PetscIJacobian.cpp.

Function Documentation

PetscErrorCode PetscIJacobian	(	TS	ts,
		PetscReal	t,
		Vec	Y,
		Vec	Ydot,
		PetscReal	a,
		Mat	A,
		Mat	B,
		void *	ctxt
	)

Compute the Jacobian for implicit time integration of the governing equations: The ODE, obtained after discretizing the governing PDE in space, is expressed as follows:

\begin{equation} \frac {d{\bf U}}{dt} = {\bf F}\left({\bf U}\right) \Rightarrow \dot{\bf U} - {\bf F}\left({\bf U}\right) = 0, \end{equation}

where \({\bf F}\) is the spatially discretized right-hand-side, and \({\bf U}\) represents the entire solution vector. The Jacobian is thus given by:

\begin{equation} {\bf J} = \left[\alpha{\bf I} - \frac {\partial {\bf F}} {\partial {\bf U}} \right] \end{equation}

where \(\alpha\) is the shift coefficient (PETScContext::shift) of the time integration method.

Matrix-free representation: The Jacobian is computed using a matrix-free approach, where the entire Jacobian is never assembled, computed, or stored. Instead, the action of the Jacobian on a vector is defined through functions (PetscJacobianFunction_JFNK() for nonlinear systems, and PetscJacobianFunction_Linear() for linear systems). This approach works well with iterative solvers. Thus, this function just does the following:

• Saves the PETScContext::shift ( \(\alpha\)) and PETScContext::waqt (current simulation time) to the application context (so that PetscJacobianFunction_JFNK() or PetscJacobianFunction_Linear() can access these values).
• If a preconditioner is being used, calls the function to compute the preconditioning matrix.

Notes:

• The Jacobian is defined as the PETSc type MatShell (https://petsc.org/release/docs/manualpages/Mat/MATSHELL.html)
• Y and Ydot in the code are \({\bf U}\) and \(\dot{\bf U}\), respectively. PETsc denotes the state vector with \({\bf Y}\) in its time integrators.
• See https://petsc.org/release/docs/manualpages/TS/index.html for documentation on PETSc's time integrators.
• All functions and variables whose names start with Vec, Mat, PC, KSP, SNES, and TS are defined by PETSc. Refer to the PETSc documentation (https://petsc.org/release/docs/). Usually, googling with the function or variable name yields the specific doc page dealing with that function/variable.

Parameters

ts	Time stepping object (see PETSc TS)
t	Current time
Y	Solution vector
Ydot	Time-derivative of solution vector
a	Shift
A	Jacobian matrix
B	Preconditioning matrix
ctxt	Application context

Definition at line 52 of file PetscIJacobian.cpp.

{
  PETScContext* context = (PETScContext*) ctxt;

  PetscFunctionBegin;
  for (int ns = 0; ns < context->nsims; ns++) {
    ((SimulationObject*)context->simobj)[ns].solver.count_IJacobian++;
  }
  context->shift = a;
  context->waqt  = t;
  /* Construct preconditioning matrix */
  if (context->flag_use_precon) PetscComputePreconMatImpl(B,Y,context);

  PetscFunctionReturn(0);
}

PetscErrorCode PetscJacobianFunction_JFNK	(	Mat	Jacobian,
		Vec	Y,
		Vec	F
	)

Computes the action of the Jacobian on a vector: See documentation for PetscIJacobian() for the definition of the Jacobian. This function computes its action on a vector for a nonlinear system by taking the directional derivative, as follows:

\begin{equation} {\bf f} = {\bf J}\left({\bf U}_0\right){\bf y} = \frac {\partial \mathcal{F}\left({\bf U}\right)} {\partial {\bf U}}\left({\bf U}_0\right) {\bf y} \approx \frac{1}{\epsilon} \left[ \mathcal{F}\left({\bf U}_0+\epsilon{\bf y}\right)-\mathcal{F}\left({\bf U}_0\right) \right] \end{equation}

In the context of the implicit time integration of the ODE given by

\begin{equation} \frac {d {\bf U}} {dt} = {\bf F}\left({\bf U}\right) \Rightarrow \frac {d {\bf U}} {dt} - {\bf F}\left({\bf U}\right) = 0, \end{equation}

we have that

\begin{equation} \mathcal{F}\left({\bf U}\right) \equiv \dot{\bf U} - {\bf F}\left({\bf U}\right) \Rightarrow {\bf J}\left({\bf U}\right) = \left[\alpha{\bf I} - \frac {\partial {\bf F}\left({\bf U}\right)} {\partial {\bf U}} \right]. \end{equation}

where \(\alpha\) (PETScContext::shift) is the shift variable specific to the time integration method. So this function computes

\begin{equation} {\bf f} = \alpha {\bf y} - \frac{1}{\epsilon} \left[ {\bf F}\left({\bf U}_0+\epsilon{\bf y}\right)-{\bf F}\left({\bf U}_0\right) \right] \end{equation}

In the code, \({\bf y}\) is Y, \(\bf f\) is F, and \({\bf U}_0\) is HyPar::uref (the reference solution at which the nonlinear Jacobian is computed). See papers on Jacobian-free Newton-Krylov (JFNK) methods to understand how \(\epsilon\) is computed.

Notes:

• For a nonlinear spatial discretization, the right-hand-side must be computed without the nonlinearity (i.e. with a previously computed or "frozen" discretization operator). This ensures that the Jacobian being computed is consistent.
• See https://petsc.org/release/docs/manualpages/TS/index.html for documentation on PETSc's time integrators.
• All functions and variables whose names start with Vec, Mat, PC, KSP, SNES, and TS are defined by PETSc. Refer to the PETSc documentation (https://petsc.org/release/docs/). Usually, googling with the function or variable name yields the specific doc page dealing with that function/variable.

Parameters

Jacobian	Jacobian matrix
Y	Input vector
F	Output vector (Jacobian times input vector)

Definition at line 111 of file PetscIJacobian.cpp.

{
  PETScContext* context(nullptr);

  PetscFunctionBegin;

  MatShellGetContext(Jacobian,&context);
  SimulationObject* sim = (SimulationObject*) context->simobj;
  int nsims = context->nsims;

  double normY;
  VecNorm(Y,NORM_2,&normY);

  if (normY < 1e-16) {

    /* F = 0 */
    VecZeroEntries(F);
    /* [J]Y = aY - F(Y) */
    VecAXPBY(F,context->shift,0,Y);

  } else {
  
    double epsilon =  context->jfnk_eps / normY;
    double t = context->waqt; /* current stage/step time */

    for (int ns = 0; ns < nsims; ns++) {

      HyPar* solver = &(sim[ns].solver);
      MPIVariables* mpi = &(sim[ns].mpi);
      solver->count_IJacFunction++;
  
      int size = solver->npoints_local_wghosts;
  
      double *u       = solver->u;
      double *uref    = solver->uref;
      double *rhsref  = solver->rhsref;
      double *rhs     = solver->rhs;
  
      /* copy solution from PETSc vector */
      TransferVecFromPETSc(u,Y,context,ns,context->offsets[ns]);
      _ArrayAYPX_(uref,epsilon,u,size*solver->nvars);
      /* apply boundary conditions and exchange data over MPI interfaces */
      solver->ApplyBoundaryConditions(solver,mpi,u,NULL,t);
      MPIExchangeBoundariesnD(  solver->ndims,
                                solver->nvars,
                                solver->dim_local,
                                solver->ghosts,
                                mpi,
                                u );
  
      /* Evaluate hyperbolic, parabolic and source terms  and the RHS for U+dU */
      _ArraySetValue_(rhs,size*solver->nvars,0.0);
      solver->HyperbolicFunction( solver->hyp,u,solver,mpi,t,0,
                                  solver->FFunction,solver->Upwind);
      _ArrayAXPY_(solver->hyp,-1.0,rhs,size*solver->nvars);
      solver->ParabolicFunction (solver->par,u,solver,mpi,t);
      _ArrayAXPY_(solver->par, 1.0,rhs,size*solver->nvars);
      solver->SourceFunction (solver->source,u,solver,mpi,t);
      _ArrayAXPY_(solver->source, 1.0,rhs,size*solver->nvars);
  
      _ArrayAXPY_(rhsref,-1.0,rhs,size*solver->nvars);
      /* Transfer RHS to PETSc vector */
      TransferVecToPETSc(rhs,F,context,ns,context->offsets[ns]);
    }

    /* [J]Y = aY - F(Y) */
    VecAXPBY(F,context->shift,(-1.0/epsilon),Y);

  }

  PetscFunctionReturn(0);
}

PetscErrorCode PetscJacobianFunction_Linear	(	Mat	Jacobian,
		Vec	Y,
		Vec	F
	)

Computes the action of the Jacobian on a vector: See documentation for PetscIJacobian() for the definition of the Jacobian. This function computes its action on a vector for a linear system by taking the directional derivative, as follows:

\begin{equation} {\bf f} = {\bf J}{\bf y} = \frac {\partial \mathcal{F}\left({\bf U}\right)} {\partial {\bf U}} {\bf y} = \left[ \mathcal{F}\left({\bf U}_0+{\bf y}\right)-\mathcal{F}\left({\bf U}_0\right) \right], \end{equation}

where \(\mathcal{F}\) is linear, and thus \({\bf J}\) is a constant ( \(\mathcal{F}\left({\bf y}\right) = {\bf J}{\bf y}\)). In the context of the implicit time integration of a linear ODE given by

\begin{equation} \frac {d {\bf U}} {dt} = {\bf F}\left({\bf U}\right) \Rightarrow \frac {d {\bf U}} {dt} - {\bf F}\left({\bf U}\right) = 0, \end{equation}

we have that

\begin{equation} \mathcal{F}\left({\bf U}\right) \equiv \dot{\bf U} - {\bf F}\left({\bf U}\right) \Rightarrow {\bf J}\left({\bf U}\right) = \left[\alpha{\bf I} - \frac {\partial {\bf F}\left({\bf U}\right)} {\partial {\bf U}} \right]. \end{equation}

where \(\alpha\) (PETScContext::shift) is the shift variable specific to the time integration method. So this function computes

\begin{equation} {\bf f} = \alpha {\bf y} - \left[ {\bf F}\left({\bf U}_0+{\bf y}\right)-{\bf F}\left({\bf U}_0\right) \right] \end{equation}

In the code, \({\bf y}\) is Y, \(\bf f\) is F, and \({\bf U}_0\) is HyPar::uref (the reference solution at which the nonlinear Jacobian is computed).

Since \(\mathcal{F}\) is linear,

\begin{equation} {\bf J}{\bf y} = \left[ \mathcal{F}\left({\bf U}_0+{\bf y}\right)-\mathcal{F}\left({\bf U}_0\right) \right] = \mathcal{F}\left({\bf y}\right). \end{equation}

However, the Jacobian is not computed as \(\mathcal{F}\left({\bf y}\right)\) because of the following reason: this function is used by the equation solver within the implicit time integrator in PETSc, and \({\bf y}\) represents the change in the solution, i.e. \(\Delta {\bf U}\), and not the solution \(\bf U\). Thus, evaluating \(\mathcal{F}\left({\bf y}\right)\) using HyPar::HyperbolicFunction, HyPar::ParabolicFunction, and HyPar::SourceFunction is incorrect since these functions expect \(\bf U\) as the input.

Notes:

• For a nonlinear spatial discretization, the right-hand-side must be computed without the nonlinearity (i.e. with a previously computed or "frozen" discretization operator). This ensures that the Jacobian being computed is consistent, and is truly linear.
• See https://petsc.org/release/docs/manualpages/TS/index.html for documentation on PETSc's time integrators.
• All functions and variables whose names start with Vec, Mat, PC, KSP, SNES, and TS are defined by PETSc. Refer to the PETSc documentation (https://petsc.org/release/docs/). Usually, googling with the function or variable name yields the specific doc page dealing with that function/variable.

Parameters

Jacobian	Jacobian matrix
Y	Input vector
F	Output vector (Jacobian times input vector

Definition at line 236 of file PetscIJacobian.cpp.

{
  PETScContext* context(nullptr);

  PetscFunctionBegin;

  MatShellGetContext(Jacobian,&context);
  SimulationObject* sim = (SimulationObject*) context->simobj;
  int nsims = context->nsims;

  double normY;
  VecNorm(Y,NORM_2,&normY);

  if (normY < 1e-16) {

    /* F = 0 */
    VecZeroEntries(F);
    /* [J]Y = aY - F(Y) */
    VecAXPBY(F,context->shift,0,Y);

  } else {
  
    double t = context->waqt; /* current stage/step time */

    for (int ns = 0; ns < nsims; ns++) {

      HyPar* solver = &(sim[ns].solver);
      MPIVariables* mpi = &(sim[ns].mpi);
      solver->count_IJacFunction++;
  
      int size = solver->npoints_local_wghosts;
  
      double *u       = solver->u;
      double *uref    = solver->uref;
      double *rhsref  = solver->rhsref;
      double *rhs     = solver->rhs;
  
      /* copy solution from PETSc vector */
      TransferVecFromPETSc(u,Y,context,ns,context->offsets[ns]);
      _ArrayAYPX_(uref,1.0,u,size*solver->nvars);
      /* apply boundary conditions and exchange data over MPI interfaces */
      solver->ApplyBoundaryConditions(solver,mpi,u,NULL,t);
      MPIExchangeBoundariesnD(  solver->ndims,
                                solver->nvars,
                                solver->dim_local,
                                solver->ghosts,
                                mpi,
                                u );
  
      /* Evaluate hyperbolic, parabolic and source terms  and the RHS for U+dU */
      _ArraySetValue_(rhs,size*solver->nvars,0.0);
      solver->HyperbolicFunction( solver->hyp,u,solver,mpi,t,0,
                                  solver->FFunction,solver->Upwind);
      _ArrayAXPY_(solver->hyp,-1.0,rhs,size*solver->nvars);
      solver->ParabolicFunction (solver->par,u,solver,mpi,t);
      _ArrayAXPY_(solver->par, 1.0,rhs,size*solver->nvars);
      solver->SourceFunction (solver->source,u,solver,mpi,t);
      _ArrayAXPY_(solver->source, 1.0,rhs,size*solver->nvars);
  
      _ArrayAXPY_(rhsref,-1.0,rhs,size*solver->nvars);
      /* Transfer RHS to PETSc vector */
      TransferVecToPETSc(rhs,F,context,ns,context->offsets[ns]);
    }

    /* [J]Y = aY - F(Y) */
    VecAXPBY(F,context->shift,-1.0,Y);

  }

  PetscFunctionReturn(0);
}