Bugfixes to support FEI Titan microscope and camera (#154)

* E-RC: added commented fix to camera client eval dict, may be needed if image is too large

* Attempts at hot-fixing beamshift calibration

* E-RC: take into account the factor of binning when tracking @ FastADT

* Revert "E-RC: take into account the factor of binning when tracking @ FastADT"

This reverts commit 24cc9fa.

* E-RC: take into account the factor of binning when tracking @ FastADT + display them properly

* E-RC: fix the coordinate order in ctrl frame toggle_rmb_beam callback

* E-RC: Refactor annotate_videostream to use binsize parameter

* E-RC: patch instamatic RPC to allow get_movie generator

* E-RC: Fixes to type passed, generator unpacking

* ER-C: remove unused socket protocol, comments, debug statements

* ER-C: allow handling generators (get_movie) by the cam server

* Update documentation with relevant changes from https://sites.google.com/view/instamatic-on-titan

* Update `docs/requirements.txt` to prevent ongoing version skew

Author: Baharis

docs/config.md

@@ -207,7 +207,7 @@ This file holds the specifications of the camera. This file is must be located t
 : Give the interface of the camera interface to connect to, for example: `timepix`/`emmenu`/`simulate`/`gatan`/`merlin`. Leave blank or set to `None` to load the camera specs, but do not load the camera module (this also turns off the videostream gui).
 
 **dead_time**
-: Set the dead time (i.e. the gap between acquisitions) of the detector; if this value (`camera.dead_time`) is not set but required, Instamatic might attempt to use `CalibMovieDelays.dead_time` value calibrated via `instamatic.calibrate_movie_delays` instead. Typically, Instamatic will not run this calibration automatically: the user needs to either set `camera.dead_time` or call `instamatic.calibrate_movie_delays` themselves.
+: Set the dead time (i.e. the gap between acquisitions) of the detector. This value is especially important for cameras what work remotely or otherwise feature dead time significant when compared to typical data collection time. If `camera.dead_time` is not set but required, Instamatic might attempt to use `CalibMovieDelays.dead_time` value calibrated via `instamatic.calibrate_movie_delays` instead. Typically, Instamatic will not run this calibration automatically: the user needs to either set `camera.dead_time` or call `instamatic.calibrate_movie_delays` themselves.
 
 **default_binsize**
 : Set the default binsize, default: `1`.

docs/network.md

@@ -40,7 +40,7 @@ use_cam_server: False
 
 ## Example 2
 
-This is an example where the microscope and camera PCs should be controlled through an intermediate support PC.
+In this example, the microscope and camera are controlled by a dedicated computer(s), distinct from an intermediate support PC running the Instamatic GUI. This scenario is preferred if, for any reason, Instamatic can not be fully installed on the Microscope/Camera PC such as when using [instamatic-tecnai-server](https://github.com/instamatic-dev/instamatic-tecnai-server) in lieu of the full Instamatic.
 
 If your camera can be controlled directly through TCP/IP, such as the MerlinEM or ASI Cheetah (via `serval`), do not use `instamatic.camserver`, but connect directly to the IP. For example, for Merlin.
 
@@ -72,6 +72,8 @@ cam_server_port: 8088
 cam_use_shared_memory: False
 ```
 
+A case of a setup where the Microscope PC supports both the TEM and a camera server via the Tecnai server while the main GUI runs on a separate Support PC is partially discussed [here](https://sites.google.com/view/instamatic-on-titan).
+
 ## Example 3
 
 If your camera cannot be controlled through TCP/IP, you might try this solution. This seems to be a common setup for TFS/FEI microscopes.

docs/requirements.txt

@@ -3,4 +3,5 @@ mkdocs
 mkdocs-jupyter
 mkdocs-gen-files
 mkdocs-material
-mkdocstrings[python]
+mkdocstrings>=0.26
+mkdocstrings-python>=1.10

docs/setup.md

@@ -8,7 +8,9 @@ If you are using a JEOL TEM, make sure `instamatic` is installed on a computer w
 
 ## FEI
 
-For FEI microscopes, `instamatic` must be installed on the microscope control PC. Alternatively, it can be installed on both the microscope PC and the camera PC, running `instamatic.temserver` on the microscope PC, and establishing a connection over the local network. See the config documentation for how to set this up.
+For FEI microscopes, `instamatic` must be installed on the microscope control PC. Alternatively, it can be installed on both the microscope PC and the camera PC, running `instamatic.temserver` on the microscope PC, and establishing a connection over the local network. If any server PC does not support modern software, [instamatic-dev/instamatic-tecnai-server](https://github.com/instamatic-dev/instamatic-tecnai-server) is a drop-in replacement that requires Python 3.4 only.
+
+See the config documentation for how to set this up.
 
 ## Development version
 
@@ -32,7 +34,7 @@ In order of priority:
 ### __2. Set up the microscope interface__  
    Go to the config directory from the first step.
 
-   In `config/settings.yaml` define the camera interface you want to use. You can use the autoconfig tool or one of the example files and modify those. You can name these files anything you want, as long as the name under `microscope` matches the filename in `config/microscope`
+   In `config/settings.yaml` define the camera interface you want to use. You can use the autoconfig tool or one of the example files and modify those. You can name these files anything you want, as long as the name under `microscope` matches the filename in `config/microscope`.
 
 ### __3. Set up the magnifications and camera lengths__  
    In the config file, i.e `config/microscope/jeol.yaml`, set the correct camera lengths (`ranges/diff`) and magnifications for your microscopes (`ranges/lowmag` and `ranges/mag1`). Also make sure you set the wavelength. Again, the autoconfig tool is your best friend, otherwise, the way to get those numbers is to simply write them down as you turn the magnification knob on the microcope.
@@ -41,10 +43,10 @@ In order of priority:
    Specify the file you want to use for the camera interface, i.e. `camera: timepix` points to `config/camera/timepix.yaml`. In this file, make sure that the interface is set to your camera type and update the numbers as specified in the config documentation. If you do not want to set up the camera interface at this moment, you can use `camera: simulate` to fake the camera connection.
 
 ### __5. Make the calibration table__  
-   For each of the magnfications defined in `config/microscope/jeol.yaml`, specify the pixel sizes in the file defined by `calibration: jeol`, corresponding to the file `calibration/jeol.yaml`. For starters, you can simply set the calibration values to 1.0.
+   For each of the magnifications defined in `config/microscope/jeol.yaml`, specify the pixel sizes in the file defined by `calibration: jeol`, corresponding to the file `calibration/jeol.yaml`. For starters, you can simply set the calibration values to 1.0.
 
 ### __6. Test if it works__  
-   Run `instamatic.temcontroller` to start a IPython shell that initializes the connection. It should run with no crashes or warnings.
+   Run `instamatic.controller` to start a IPython shell that initializes the connection. It should run with no crashes or warnings.
 
 ### __7. Update `settings.yaml`__  
    There are a few more choices to make in `instamatic/settings.yaml`. If you use a TVIPS camera, make sure you put `use_cam_server: true`.

readme.md

@@ -22,11 +22,12 @@ Cameras supported:
 - ASI CheeTah through `serval-toolkit` library
 - TVIPS cameras through EMMENU4 API
 - Quantum Detectors MerlinEM
+- Gatan cameras through FEI scripting interface
 - (Gatan cameras through DM plugin [1])
 
 Instamatic has been developed on a JEOL-2100 with a Timepix camera, and a JEOL-1400 and JEOL-3200 with TVIPS cameras (XF416/F416).
 
-See [instamatic-dev/instamatic-tecnai-server](https://github.com/instamatic-dev/instamatic-tecnai-server) for a TEM interface to control a FEI Tecnai-TEM on Windows XP/Python 3.4 via instamatic.
+See [instamatic-dev/instamatic-tecnai-server](https://github.com/instamatic-dev/instamatic-tecnai-server) for a TEM interface to control a FEI Tecnai or FEI Titan TEM and associated cameras on Windows XP/Python 3.4 via instamatic.
 
 [1]: Support for Gatan cameras is somewhat underdeveloped. As an alternative, a DigitalMicrograph script for collecting cRED data on a OneView camera (or any other Gatan camera) can be found [here](https://github.com/instamatic-dev/InsteaDMatic).
 
@@ -47,7 +48,7 @@ pip install instamatic
 
 ## OS requirement
 
-The package requires Windows 7 or higher. It has been mainly developed and tested under windows 7 and higher.
+The package requires Windows 7 or higher. It has been mainly developed and tested under Windows 7 and higher.
 
 ## Package dependencies
 

src/instamatic/calibrate/calibrate_beamshift.py

@@ -101,7 +101,8 @@ def live(
     ) -> Self:
         while True:
             c = calibrate_beamshift(ctrl=ctrl, save_images=True, outdir=outdir)
-            with c.annotate_videostream(vsp) if vsp else nullcontext():
+            binsize = ctrl.cam.default_binsize
+            with c.annotate_videostream(vsp, binsize) if vsp else nullcontext():
                 if input(' >> Accept? [y/n] ') == 'y':
                     return c
 
@@ -127,15 +128,15 @@ def plot(self, to_file: Optional[AnyPath] = None):
             plt.show()
 
     @contextmanager
-    def annotate_videostream(self, vsp: Optional[VideoStreamProcessor] = None) -> None:
+    def annotate_videostream(self, vsp: VideoStreamProcessor, binsize: int = 1) -> None:
         shifts = np.dot(self.shifts, np.linalg.inv(self.transform))
         ins: list[DeferredImageDraw.Instruction] = []
 
         vsp.temporary_frame = np.max(self.images, axis=0)
         print('Determined (blue) vs calibrated (orange) beam positions:')
         for p, s in zip(self.pixels, shifts):
-            p = (p + self.reference_pixel)[::-1]  # xy coords inverted for plot
-            s = (s + self.reference_pixel)[::-1]  # xy coords inverted for plot
+            p = (p + self.reference_pixel)[::-1] / binsize  # xy coords inverted for plot
+            s = (s + self.reference_pixel)[::-1] / binsize  # xy coords inverted for plot
             ins.append(vsp.draw.circle(p, radius=3, fill='blue'))
             ins.append(vsp.draw.circle(s, radius=3, fill='orange'))
         ins.append(vsp.draw.circle(self.reference_pixel[::-1], radius=3, fill='black'))

src/instamatic/camera/camera_client.py

@@ -6,6 +6,7 @@
 import threading
 import time
 from functools import wraps
+from typing import Any, Generator
 
 import numpy as np
 
@@ -126,11 +127,7 @@ def _eval_dct(self, dct):
         with self._eval_lock:
             self.s.send(dumper(dct))
 
-            acquiring_image = dct['attr_name'] == 'get_image'
-            acquiring_movie = dct['attr_name'] == 'get_movie'
-
-            if acquiring_movie:
-                raise NotImplementedError('Acquiring movies over a socket is not supported.')
+            acquiring_image = dct['attr_name'] in {'get_image', 'get_movie', '__gen_next__'}
 
             if acquiring_image and not self.use_shared_memory:
                 response = self.s.recv(self._imagebufsize)
@@ -146,6 +143,8 @@ def _eval_dct(self, dct):
                 data = self.get_data_from_shared_memory(**data)
 
             if status == 200:
+                if isinstance(data, dict) and '__generator__' in data:
+                    return self._wrap_remote_generator(data['__generator__'])
                 return data
 
             elif status == 500:
@@ -206,3 +205,20 @@ def block(self):
 
     def unblock(self):
         raise NotImplementedError('This camera cannot be streamed.')
+
+    def _wrap_remote_generator(self, gen_id: str) -> Generator[Any]:
+        """Pass a reference to yield from a remote __generator__ with id."""
+
+        def generator():
+            kwargs = {'id': gen_id}
+            try:
+                while True:
+                    dct = {'attr_name': '__gen_next__', 'kwargs': kwargs}
+                    value = self._eval_dct(dct)
+                    if value is None:
+                        return
+                    yield value
+            finally:
+                self._eval_dct({'attr_name': '__gen_close__', 'kwargs': kwargs})
+
+        return generator()

src/instamatic/camera/videostream.py

@@ -72,8 +72,8 @@ def run(self):
             if self.acquireInitiateEvent.is_set():
                 r = self.request
                 self.acquireInitiateEvent.clear()
-                e = r.exposure if r.exposure else self.default_exposure
-                b = r.binsize if r.binsize else self.default_binsize
+                e = float(r.exposure if r.exposure else self.default_exposure)
+                b = int(r.binsize if r.binsize else self.default_binsize)
                 if isinstance(r, ImageRequest):
                     media = self.cam.get_image(exposure=e, binsize=b)
                     self.callback(media, request=r)

src/instamatic/experiments/fast_adt/experiment.py

@@ -213,6 +213,7 @@ def __init__(
         self.flatfield = flatfield
         self.fast_adt_frame = experiment_frame
         self.beamshift: Optional[CalibBeamShift] = None
+        self.binsize: int = 1
         self.camera_length: int = 0
 
         if videostream_frame is not None:
@@ -337,6 +338,7 @@ def start_collection(self, **params) -> None:
 
         with self.ctrl.beam.blanked(), self.ctrl.cam.blocked():
             if params['tracking_algo'] == 'manual':
+                self.binsize = self.ctrl.cam.default_binsize
                 self.runs.tracking = TrackingRun.from_params(params)
                 self.determine_pathing_manually()
             for pathing_run in self.runs.pathing:
@@ -366,8 +368,8 @@ def displayed_pathing(self, step: Step) -> None:
         draw = self.videostream_processor.draw
         instructions: list[draw.Instruction] = []
         for run_i, p in enumerate(self.runs.pathing):
-            x = p.table.at[step.Index, 'beampixel_x']
-            y = p.table.at[step.Index, 'beampixel_y']
+            x = p.table.at[step.Index, 'beampixel_x'] / self.binsize
+            y = p.table.at[step.Index, 'beampixel_y'] / self.binsize
             instructions.append(draw.circle((x, y), fill='white', radius=5))
             instructions.append(draw.circle((x, y), fill=get_color(run_i), radius=3))
         try:
@@ -388,7 +390,7 @@ def determine_pathing_manually(self) -> None:
             self.beamshift = self.get_beamshift()
             self.msg1('Locate the beam (move it if needed) and click on its center.')
             with self.click_listener as cl:
-                obs_beampixel_xy = np.array(cl.get_click().xy)
+                obs_beampixel_xy = np.array(cl.get_click().xy) * self.binsize
         cal_beampixel_yx = self.beamshift.beamshift_to_pixelcoord(self.ctrl.beamshift.get())
 
         self.ctrl.restore('FastADT_track')
@@ -401,11 +403,12 @@ def determine_pathing_manually(self) -> None:
                 self.msg1(f'Click on tracked point: {step.summary}.')
                 with self.displayed_pathing(step=step), self.click_listener:
                     click = self.click_listener.get_click()
-                delta_yx = (np.array(click.xy) - obs_beampixel_xy)[::-1]
+                click_xy = np.array(click.xy) * self.binsize
+                delta_yx = (click_xy - obs_beampixel_xy)[::-1]
                 click_beampixel_yx = cast(Sequence[float], cal_beampixel_yx + delta_yx)
                 click_beamshift_xy = self.beamshift.pixelcoord_to_beamshift(click_beampixel_yx)
                 cols = ['beampixel_x', 'beampixel_y', 'beamshift_x', 'beamshift_y']
-                run.table.loc[step.Index, cols] = *click.xy, *click_beamshift_xy
+                run.table.loc[step.Index, cols] = *click_xy, *click_beamshift_xy
                 tracking_frames.append(step.image)
             if 'image' not in run.table:
                 run.table['image'] = tracking_frames

src/instamatic/gui/ctrl_frame.py

@@ -364,9 +364,13 @@ def toggle_rmb_beam(self, _name, _index, _mode) -> None:
             self.var_rmb_beam.set(False)
             return
 
+        binning = self.ctrl.cam.default_binsize
+
         def _callback(click: ClickEvent) -> None:
             if click.button == MouseButton.RIGHT:
-                bs = calib_beamshift.pixelcoord_to_beamshift((click.y, click.x))
+                pixel_x = click.x * binning
+                pixel_y = click.y * binning
+                bs = calib_beamshift.pixelcoord_to_beamshift((pixel_y, pixel_x))
                 self.ctrl.beamshift.set(*[float(b) for b in bs])
 
         d.add_listener('rmb_beam', _callback, active=True)