connectors/ConnectorBase.py
author Edouard Tisserant
Thu, 18 Apr 2019 14:35:42 +0200
branchfix_PLC_runtime_shutdown
changeset 2604 c8a25a3a7f8b
parent 2542 a3ec35ee94e7
child 2621 af09744a468e
permissions -rw-r--r--
Runtime worker : unblock the last waiting job and prevent any new job to wait when Main Worker is being shut down.
2463
8742337a9fe3 Chunk based transfer for PLC binary and extra files, and some collateral code refactoring.
Edouard Tisserant
parents:
diff changeset
     1
#!/usr/bin/env python
8742337a9fe3 Chunk based transfer for PLC binary and extra files, and some collateral code refactoring.
Edouard Tisserant
parents:
diff changeset
     2
# -*- coding: utf-8 -*-
8742337a9fe3 Chunk based transfer for PLC binary and extra files, and some collateral code refactoring.
Edouard Tisserant
parents:
diff changeset
     3
8742337a9fe3 Chunk based transfer for PLC binary and extra files, and some collateral code refactoring.
Edouard Tisserant
parents:
diff changeset
     4
# See COPYING file for copyrights details.
8742337a9fe3 Chunk based transfer for PLC binary and extra files, and some collateral code refactoring.
Edouard Tisserant
parents:
diff changeset
     5
2492
7dd551ac2fa0 check_sources.sh makes me become even less productive
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 2487
diff changeset
     6
from __future__ import absolute_import
2540
fca79ca84272 Replace md5 module with hashlib
Andrey Skvortsov <andrej.skvortzov@gmail.com>
parents: 2492
diff changeset
     7
import hashlib
2463
8742337a9fe3 Chunk based transfer for PLC binary and extra files, and some collateral code refactoring.
Edouard Tisserant
parents:
diff changeset
     8
2492
7dd551ac2fa0 check_sources.sh makes me become even less productive
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 2487
diff changeset
     9
2463
8742337a9fe3 Chunk based transfer for PLC binary and extra files, and some collateral code refactoring.
Edouard Tisserant
parents:
diff changeset
    10
class ConnectorBase(object):
8742337a9fe3 Chunk based transfer for PLC binary and extra files, and some collateral code refactoring.
Edouard Tisserant
parents:
diff changeset
    11
2465
47d3aea2be30 Bigger chunks, from 16k to 1M
Edouard Tisserant
parents: 2463
diff changeset
    12
    chuncksize = 1024*1024
2492
7dd551ac2fa0 check_sources.sh makes me become even less productive
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 2487
diff changeset
    13
2487
6a4f9a061994 Reworked chunk based transfer to support duplicated files (i.e. files with same content, but different names)
Edouard Tisserant
parents: 2465
diff changeset
    14
    def BlobFromFile(self, filepath, seed):
2540
fca79ca84272 Replace md5 module with hashlib
Andrey Skvortsov <andrej.skvortzov@gmail.com>
parents: 2492
diff changeset
    15
        s = hashlib.new('md5')
2487
6a4f9a061994 Reworked chunk based transfer to support duplicated files (i.e. files with same content, but different names)
Edouard Tisserant
parents: 2465
diff changeset
    16
        s.update(seed)
6a4f9a061994 Reworked chunk based transfer to support duplicated files (i.e. files with same content, but different names)
Edouard Tisserant
parents: 2465
diff changeset
    17
        blobID = self.SeedBlob(seed)
2463
8742337a9fe3 Chunk based transfer for PLC binary and extra files, and some collateral code refactoring.
Edouard Tisserant
parents:
diff changeset
    18
        with open(filepath, "rb") as f:
2487
6a4f9a061994 Reworked chunk based transfer to support duplicated files (i.e. files with same content, but different names)
Edouard Tisserant
parents: 2465
diff changeset
    19
            while blobID == s.digest():
2492
7dd551ac2fa0 check_sources.sh makes me become even less productive
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 2487
diff changeset
    20
                chunk = f.read(self.chuncksize)
7dd551ac2fa0 check_sources.sh makes me become even less productive
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 2487
diff changeset
    21
                if len(chunk) == 0:
7dd551ac2fa0 check_sources.sh makes me become even less productive
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 2487
diff changeset
    22
                    return blobID
2463
8742337a9fe3 Chunk based transfer for PLC binary and extra files, and some collateral code refactoring.
Edouard Tisserant
parents:
diff changeset
    23
                blobID = self.AppendChunkToBlob(chunk, blobID)
8742337a9fe3 Chunk based transfer for PLC binary and extra files, and some collateral code refactoring.
Edouard Tisserant
parents:
diff changeset
    24
                s.update(chunk)