Merge branch 'master' of github.com:ndokter/dsmr_parser

Improve documentation asyncio

.github/workflows/tests.yml

@@ -12,10 +12,11 @@ jobs:
     strategy:
       matrix:
         python-version:
-          - 3.6
-          - 3.7
-          - 3.8
-          - 3.9
+          - '3.6'
+          - '3.7'
+          - '3.8'
+          - '3.9'
+          - '3.10'
 
     name: Python ${{ matrix.python-version }}
     steps:

CHANGELOG.rst

@@ -1,6 +1,13 @@
 Change Log
 ----------
 
+**0.33** (2022-04-20)
+
+- Test Python 3.10 in CI + legacy badge fix (`pull request #105 <https://github.com/ndokter/dsmr_parser/pull/105>`_).
+- Update telegram_specifications.py (`pull request #106 <https://github.com/ndokter/dsmr_parser/pull/106>`_).
+- Improve compatiblity with Belgian standard (`pull request #107 <https://github.com/ndokter/dsmr_parser/pull/107>`_).
+- Improve documentation asyncio (`pull request #63 <https://github.com/ndokter/dsmr_parser/pull/63>`_).
+
 **0.32** (2022-01-04)
 
 - Support DSMR data read via RFXtrx with integrated P1 reader (`pull request #98 <https://github.com/ndokter/dsmr_parser/pull/98>`_).

README.rst

@@ -1,14 +1,11 @@
-**Notice:** this repository is in need of a new maintainer. If you are interested or have ideas about this, please let me know.
-
-
 DSMR Parser
 ===========
 
 .. image:: https://img.shields.io/pypi/v/dsmr-parser.svg
     :target: https://pypi.python.org/pypi/dsmr-parser
 
-.. image:: https://travis-ci.org/ndokter/dsmr_parser.svg?branch=master
-    :target: https://travis-ci.org/ndokter/dsmr_parser
+.. image:: https://img.shields.io/github/workflow/status/ndokter/dsmr_parser/Tests/master
+    :target: https://github.com/ndokter/dsmr_parser/actions/workflows/tests.yml
 
 A library for parsing Dutch Smart Meter Requirements (DSMR) telegram data. It
 also includes client implementation to directly read and parse smart meter data.
@@ -17,7 +14,7 @@ also includes client implementation to directly read and parse smart meter data.
 Features
 --------
 
-DSMR Parser supports DSMR versions 2, 3, 4 and 5. It has been tested with Python 3.5, 3.6, 3.7, 3.8 and 3.9.
+DSMR Parser supports DSMR versions 2, 3, 4 and 5. See for the `currently supported/tested Python versions here <https://github.com/ndokter/dsmr_parser/blob/master/.github/workflows/tests.yml#L14>`_.
 
 
 Client module usage
@@ -42,10 +39,6 @@ process because the code is blocking (not asynchronous):
      for telegram in serial_reader.read():
          print(telegram)  # see 'Telegram object' docs below
 
-**AsyncIO client**
-
-To be documented.
-
 **Socket client**
 
 Read a remote serial port (for example using ser2net) and work with the parsed telegrams.
@@ -65,6 +58,117 @@ It should be run in a separate process because the code is blocking (not asynchr
      for telegram in socket_reader.read():
          print(telegram)  # see 'Telegram object' docs below
 
+**AsyncIO client**
+
+For a test run using a tcp server (lasting 20 seconds) use the following example:
+
+.. code-block:: python
+
+    import asyncio
+    import logging
+    from dsmr_parser import obis_references
+    from dsmr_parser.clients.protocol import create_dsmr_reader, create_tcp_dsmr_reader
+
+    logging.basicConfig(level=logging.INFO, format='%(message)s')
+
+    HOST = MY_HOST
+    PORT = MY_PORT
+    DSMR_VERSION = MY_DSMR_VERSION
+
+    logger = logging.getLogger('tcpclient')
+    logger.debug("Logger created")
+
+    def printTelegram(telegram):
+        logger.info(telegram)
+
+
+    async def main():
+        try:
+            logger.debug("Getting loop")
+            loop = asyncio.get_event_loop()
+            logger.debug("Creating reader")
+            await 	create_tcp_dsmr_reader(
+                                HOST,
+                                PORT,
+                                DSMR_VERSION,
+                                printTelegram,
+                                loop
+                                )
+            logger.debug("Reader created going to sleep now")					
+            await asyncio.sleep(20)
+            logger.info('Finished run')					
+        except Exception as e:
+            logger.error("Unexpected error: "+ e)
+
+    asyncio.run(main())
+
+Note the creation of a callback function to call when a telegram is received. In this case `printTelegram`. Normally the used loop is the one running.
+
+Currently the asyncio implementation does not support returning telegram objects directly as a `read_as_object()` for async tcp is currently not implemented.
+Moreover, the telegram passed to `telegram_callback(telegram)` is already parsed. Therefore we can't feed it into the telegram constructor directly as that expects unparsed telegrams
+
+However, if we construct a mock TelegramParser that just returns the already parsed object we can work around this. An example is below:
+
+.. code-block:: python
+
+    import asyncio
+    import logging
+    #from dsmr_parser import obis_references
+    #from dsmr_parser import telegram_specifications
+    #from dsmr_parser.clients.protocol import create_dsmr_reader, create_tcp_dsmr_reader
+    #from dsmr_parser.objects import Telegram
+
+    logging.basicConfig(level=logging.INFO, format='%(message)s')
+
+    HOST = MY_HOST
+    PORT = MY_PORT
+    DSMR_VERSION = MY_DSMR_VERSION
+
+    logger = logging.getLogger('tcpclient')
+    logger.debug("Logger created")
+
+    class mockTelegramParser(object):
+
+        def parse(self, telegram):
+            return telegram
+
+    telegram_parser = mockTelegramParser()
+
+    def printTelegram(telegram):
+        try:
+            logger.info(Telegram(telegram, telegram_parser, telegram_specifications.V4))
+        except InvalidChecksumError as e:
+            logger.warning(str(e))
+        except ParseError as e:
+            logger.error('Failed to parse telegram: %s', e)
+
+
+    async def main():
+        try:
+            logger.debug("Getting loop")
+            loop = asyncio.get_event_loop()
+            logger.debug("Creating reader")
+            await 	create_tcp_dsmr_reader(
+                                HOST,
+                                PORT,
+                                DSMR_VERSION,
+                                printTelegram,
+                                loop
+                                )
+            logger.debug("Reader created going to sleep now")					
+            while True:
+                await asyncio.sleep(1)
+        except Exception as e:
+            logger.error("Unexpected error: "+ e)
+            raise
+
+    if __name__ == '__main__':
+        try:
+            asyncio.run(main())
+        except (KeyboardInterrupt, SystemExit):
+            logger.info('Closing down...')					
+        except Exception as e:
+            logger.error("Unexpected error: "+ e)
 
 Parsing module usage
 --------------------
@@ -73,39 +177,39 @@ into a dictionary.
 
 .. code-block:: python
 
-     from dsmr_parser import telegram_specifications
-     from dsmr_parser.parsers import TelegramParser
+    from dsmr_parser import telegram_specifications
+    from dsmr_parser.parsers import TelegramParser
 
-     # String is formatted in separate lines for readability.
-     telegram_str = (
-         '/ISk5\\2MT382-1000\r\n'
-         '\r\n'
-         '0-0:96.1.1(4B384547303034303436333935353037)\r\n'
-         '1-0:1.8.1(12345.678*kWh)\r\n'
-         '1-0:1.8.2(12345.678*kWh)\r\n'
-         '1-0:2.8.1(12345.678*kWh)\r\n'
-         '1-0:2.8.2(12345.678*kWh)\r\n'
-         '0-0:96.14.0(0002)\r\n'
-         '1-0:1.7.0(001.19*kW)\r\n'
-         '1-0:2.7.0(000.00*kW)\r\n'
-         '0-0:17.0.0(016*A)\r\n'
-         '0-0:96.3.10(1)\r\n'
-         '0-0:96.13.1(303132333435363738)\r\n'
-         '0-0:96.13.0(303132333435363738393A3B3C3D3E3F303132333435363738393A3B3C3D3E'
-         '3F303132333435363738393A3B3C3D3E3F303132333435363738393A3B3C3D3E3F30313233'
-         '3435363738393A3B3C3D3E3F)\r\n'
-         '0-1:96.1.0(3232323241424344313233343536373839)\r\n'
-         '0-1:24.1.0(03)\r\n'
-         '0-1:24.3.0(090212160000)(00)(60)(1)(0-1:24.2.1)(m3)\r\n'
-         '(00001.001)\r\n'
-         '0-1:24.4.0(1)\r\n'
-         '!\r\n'
-     )
+    # String is formatted in separate lines for readability.
+    telegram_str = (
+        '/ISk5\\2MT382-1000\r\n'
+        '\r\n'
+        '0-0:96.1.1(4B384547303034303436333935353037)\r\n'
+        '1-0:1.8.1(12345.678*kWh)\r\n'
+        '1-0:1.8.2(12345.678*kWh)\r\n'
+        '1-0:2.8.1(12345.678*kWh)\r\n'
+        '1-0:2.8.2(12345.678*kWh)\r\n'
+        '0-0:96.14.0(0002)\r\n'
+        '1-0:1.7.0(001.19*kW)\r\n'
+        '1-0:2.7.0(000.00*kW)\r\n'
+        '0-0:17.0.0(016*A)\r\n'
+        '0-0:96.3.10(1)\r\n'
+        '0-0:96.13.1(303132333435363738)\r\n'
+        '0-0:96.13.0(303132333435363738393A3B3C3D3E3F303132333435363738393A3B3C3D3E'
+        '3F303132333435363738393A3B3C3D3E3F303132333435363738393A3B3C3D3E3F30313233'
+        '3435363738393A3B3C3D3E3F)\r\n'
+        '0-1:96.1.0(3232323241424344313233343536373839)\r\n'
+        '0-1:24.1.0(03)\r\n'
+        '0-1:24.3.0(090212160000)(00)(60)(1)(0-1:24.2.1)(m3)\r\n'
+        '(00001.001)\r\n'
+        '0-1:24.4.0(1)\r\n'
+        '!\r\n'
+    )
 
-     parser = TelegramParser(telegram_specifications.V3)
-    
-     telegram = parser.parse(telegram_str)
-     print(telegram)  # see 'Telegram object' docs below
+    parser = TelegramParser(telegram_specifications.V3)
+
+    telegram = parser.parse(telegram_str)
+    print(telegram)  # see 'Telegram object' docs below
 
 Telegram dictionary
 -------------------
@@ -158,7 +262,7 @@ Example to get some of the values:
      gas_reading = telegram[obis_references.HOURLY_GAS_METER_READING]
 
     # See dsmr_reader.obis_references for all readable telegram values.
-    # Note that the avilable values differ per DSMR version.
+    # Note that the available values differ per DSMR version.
 
 Telegram as an Object
 ---------------------

dsmr_parser/clients/filereader.py

@@ -64,8 +64,11 @@ class FileReader(object):
         with open(self._file, "rb") as file_handle:
             while True:
                 data = file_handle.readline()
-                str = data.decode()
-                self.telegram_buffer.append(str)
+
+                if not data:
+                    break
+
+                self.telegram_buffer.append(data.decode())
 
                 for telegram in self.telegram_buffer.get_all():
                     try:

dsmr_parser/obis_name_mapping.py

@@ -50,7 +50,9 @@ EN = {
     obis.ACTUAL_TRESHOLD_ELECTRICITY: 'ACTUAL_TRESHOLD_ELECTRICITY',
     obis.ACTUAL_SWITCH_POSITION: 'ACTUAL_SWITCH_POSITION',
     obis.VALVE_POSITION_GAS: 'VALVE_POSITION_GAS',
-    obis.BELGIUM_HOURLY_GAS_METER_READING: 'BELGIUM_HOURLY_GAS_METER_READING',
+    obis.BELGIUM_5MIN_GAS_METER_READING: 'BELGIUM_5MIN_GAS_METER_READING',
+    obis.BELGIUM_MAX_POWER_PER_PHASE: 'BELGIUM_MAX_POWER_PER_PHASE',
+    obis.BELGIUM_MAX_CURRENT_PER_PHASE: 'BELGIUM_MAX_CURRENT_PER_PHASE',
     obis.LUXEMBOURG_EQUIPMENT_IDENTIFIER: 'LUXEMBOURG_EQUIPMENT_IDENTIFIER',
     obis.Q3D_EQUIPMENT_IDENTIFIER: 'Q3D_EQUIPMENT_IDENTIFIER',
     obis.Q3D_EQUIPMENT_STATE: 'Q3D_EQUIPMENT_STATE',

dsmr_parser/obis_references.py

@@ -65,7 +65,9 @@ ELECTRICITY_IMPORTED_TOTAL = r'\d-\d:1\.8\.0.+?\r\n'  # Total imported energy re
 ELECTRICITY_EXPORTED_TOTAL = r'\d-\d:2\.8\.0.+?\r\n'  # Total exported energy register (P-)
 
 # International non generalized additions (country specific) / risk for necessary refactoring
-BELGIUM_HOURLY_GAS_METER_READING = r'\d-\d:24\.2\.3.+?\r\n'  # Different code, same format.
+BELGIUM_5MIN_GAS_METER_READING = r'\d-\d:24\.2\.3.+?\r\n'  # Different code, same format.
+BELGIUM_MAX_POWER_PER_PHASE = r'\d-\d:17\.0\.0.+?\r\n'  # Applicable when power limitation is active
+BELGIUM_MAX_CURRENT_PER_PHASE = r'\d-\d:31\.4\.0.+?\r\n'  # Applicable when current limitation is active
 LUXEMBOURG_EQUIPMENT_IDENTIFIER = r'\d-\d:42\.0\.0.+?\r\n'  # Logical device name
 Q3D_EQUIPMENT_IDENTIFIER = r'\d-\d:0\.0\.0.+?\r\n'  # Logical device name
 Q3D_EQUIPMENT_STATE = r'\d-\d:96\.5\.5.+?\r\n'  # Device state (hexadecimal)

dsmr_parser/telegram_specifications.py

@@ -144,10 +144,14 @@ ALL = (V2_2, V3, V4, V5)
 
 BELGIUM_FLUVIUS = deepcopy(V5)
 BELGIUM_FLUVIUS['objects'].update({
-    obis.BELGIUM_HOURLY_GAS_METER_READING: MBusParser(
+    obis.BELGIUM_5MIN_GAS_METER_READING: MBusParser(
         ValueParser(timestamp),
         ValueParser(Decimal)
-    )
+    ),
+    obis.BELGIUM_MAX_POWER_PER_PHASE: CosemParser(ValueParser(Decimal)),
+    obis.BELGIUM_MAX_CURRENT_PER_PHASE: CosemParser(ValueParser(Decimal)),
+    obis.ACTUAL_SWITCH_POSITION: CosemParser(ValueParser(str)),
+    obis.VALVE_POSITION_GAS: CosemParser(ValueParser(str)),
 })
 
 LUXEMBOURG_SMARTY = deepcopy(V5)

setup.py

@@ -7,7 +7,7 @@ setup(
     author_email='nigel@nldr.net',
     license='MIT',
     url='https://github.com/ndokter/dsmr_parser',
-    version='0.32',
+    version='0.33',
     packages=find_packages(exclude=('test', 'test.*')),
     install_requires=[
         'pyserial>=3,<4',

test/test_filereader.py

@@ -0,0 +1,21 @@
+import unittest
+import tempfile
+
+from dsmr_parser.clients.filereader import FileReader
+from dsmr_parser.telegram_specifications import V5
+from test.example_telegrams import TELEGRAM_V5
+
+
+class FileReaderTest(unittest.TestCase):
+    def test_read_as_object(self):
+        with tempfile.NamedTemporaryFile() as file:
+            with open(file.name, "w") as f:
+                f.write(TELEGRAM_V5)
+
+            telegrams = []
+            reader = FileReader(file=file.name, telegram_specification=V5)
+            # Call
+            for telegram in reader.read_as_object():
+                telegrams.append(telegram)
+
+            self.assertEqual(len(telegrams), 1)