In waterjet and laser milling, material is removed from a solid surface in a succession of layers to create a new shape, in a depth-controlled manner. The inverse problem consists of defining the control parameters, in particular, the two-dimensional beam path, to arrive at a prescribed freeform surface. Waterjet milling (WJM) and pulsed laser ablation (PLA) are studied in this paper, since a generic nonlinear material removal model is appropriate for both of these processes. The inverse problem is usually solved for this kind of process by simply controlling dwell time in proportion to the required depth of milling at a sequence of pixels on the surface. However, this approach is only valid when shallow surfaces are etched, since it does not take into account either the footprint of the beam or its overlapping on successive passes. A discrete adjoint algorithm is proposed in this paper to improve the solution. Nonlinear effects and non-straight passes are included in the optimization, while the calculation of the Jacobian matrix does not require large computation times. Several tests are performed to validate the proposed method and the results show that tracking error is reduced typically by a factor of two in comparison to the pixel-by-pixel approach and the classical raster path strategy with straight passes. The tracking error can be as low as 2–5% and 1–2% for WJM and PLA, respectively, depending on the complexity of the target surface.