Advanced Features in PyMAD-NG

This section covers some of the most powerful capabilities in PyMAD-NG. These features allow you to create scalable and complex accelerator workflows by combining the performance of MAD-NG with Python’s expressiveness.

---

Understanding _last[] Temporary Variables

In MAD-NG, when a command returns a value, it is not automatically captured unless explicitly assigned. PyMAD-NG handles this by assigning results to a set of reserved variables: _last[1], _last[2], etc.

These are managed internally by PyMAD-NG using a helper class madp_last.last_counter, and accessed in Python via references. This allows expressions like:

result = mad.math.sqrt(2) + mad.math.log(10)

Behind the scenes, each intermediate operation is stored in a new _last[i] reference, then combined. You can access or evaluate the result using .eval():

print(result.eval())

These temporary variables are recycled unless manually stored using:

mad["my_var"] = result

This is particularly useful in expressions, multi-step computations, and avoiding naming clutter.

---

Function and Object References in MAD-NG

In PyMAD-NG, accessing or calling any MAD-NG function or object returns a Python reference to that MAD-NG entity, rather than immediately executing or resolving it. This enables symbolic chaining and precise control over execution.

Example:

r = mad.math.exp(1)
print(type(r))  # MadRef
print(r.eval())  # 2.718...

You can delay evaluation until needed, allowing reuse:

mad["result"] = mad.math.log(10) + mad.math.sin(1)

This keeps Python responsive and lets MAD-NG do the heavy lifting.

---

Real-Time Feedback with Python During Matching

MAD-NG supports callbacks and iterative evaluations, which can be tied into Python logic. One common use is during match procedures, where you want to receive intermediate updates.

Example Workflow:

In MAD:

function twiss_and_send()
  local tbl, flow = twiss {sequence=seq}
  py:send({tbl.s, tbl.beta11})
  return tbl, flow
end

In Python:

mad.match(
  command=mad.twiss_and_send,
  variables=[...],
  equalities=[...],
  objective={"fmin": 1e-3},
  maxcall=100
)

while True:
    data = mad.recv()
    if data is None:
        break
    update_plot(data)

This is ideal for live visualisation, feedback loops, or diagnostics during optimisation.

---

Using PyMAD-NG with Multiprocessing

Because PyMAD-NG communicates with MAD-NG via pipes (not shared memory), you can launch multiple independent MAD processes using os.fork() or multiprocessing.

When to Use This:

• Run parallel simulations or parameter scans

• Avoid reloading large sequences repeatedly

Example:

import os
if os.fork() == 0:
    mad = MAD()
    mad.send("... long running setup ...")
    os._exit(0)

Each process maintains its own MAD instance and data pipeline.

---

Loading and Using External MAD Files and Modules

MAD-X and MAD-NG models often consist of .seq, .mad, .madx, or .str files. You can load these via the high-level interface:

mad.MADX.load("'lhc.seq'", "'lhc.mad'")
mad.load("MADX", "lhcb1")

Or load additional MAD-NG modules:

mad.load("MAD.gphys", "melmcol")

This loads extended libraries for magnet properties, tracking models, or optics algorithms.

---

Exporting Results for External Use

After running a Twiss or Survey, the results are stored in an mtable, which can be exported to a TFS file:

mad.tbl.write("'results.tfs'", mad.quote_strings(["s", "beta11", "mu1"]))

You can read this file with tfs-pandas or use it as input to another tool.

---

Combining with NumPy and Pandas

PyMAD-NG integrates cleanly with Python’s data ecosystem:

• Pass numpy arrays to MAD-NG using MAD.send()

• Use to_df() on MAD tables to get Pandas DataFrames

• Use tfs-pandas for rich metadata support

Example:

import numpy as np
mad.send("my_array = py:recv()")
mad.send(np.linspace(0, 1, 100))

This allows direct use of scientific computation tools in tandem with accelerator modelling.

---

Managing Larger Workflows

PyMAD-NG supports:

• Loading full files with mad.loadfile("mysetup.mad")

• Organising expressions using Python variables

• Retaining command history using:

print(mad.history())

For clean resource management, always use context blocks:

with MAD() as mad:
    mad.MADX.load("'lhc.seq'", "'lhc.mad'")

This ensures the MAD process is correctly shut down when finished.

---

Summary of Advanced Features

Feature	Purpose
_last[] Variables	Track intermediate return values symbolically
Reference Objects	Access MAD-NG objects with delayed evaluation
Matching Feedback	Monitor intermediate results during match
Multiprocessing	Run multiple MAD-NG simulations in parallel
File and Module Loading	Import sequences, optics files, and Lua modules
Table Export	Write TFS files from MAD tables
NumPy / Pandas Interoperability	Pass data between Python and MAD-NG seamlessly
Project Structuring	Use MAD.loadfile(), MAD.history(), and with block

These tools are designed to give you complete control over your simulations while staying fast and maintainable.

Next: head over to Debugging & Troubleshooting to diagnose and resolve common issues in real-world workflows.