connectors/ConnectorBase.py
author Edouard Tisserant <edouard.tisserant@gmail.com>
Mon, 11 Mar 2019 17:20:15 +0100
changeset 2522 1378c18402c3
parent 2492 7dd551ac2fa0
child 2540 fca79ca84272
permissions -rw-r--r--
Fixed more pylint reports :
connectors/__init__.py:31: [W1652(deprecated-types-field), ] Accessing a deprecated fields on the types module
in fact, it should have been unused import.
2463
8742337a9fe3 Chunk based transfer for PLC binary and extra files, and some collateral code refactoring.
Edouard Tisserant
parents:
diff changeset
     1
#!/usr/bin/env python
8742337a9fe3 Chunk based transfer for PLC binary and extra files, and some collateral code refactoring.
Edouard Tisserant
parents:
diff changeset
     2
# -*- coding: utf-8 -*-
8742337a9fe3 Chunk based transfer for PLC binary and extra files, and some collateral code refactoring.
Edouard Tisserant
parents:
diff changeset
     3
8742337a9fe3 Chunk based transfer for PLC binary and extra files, and some collateral code refactoring.
Edouard Tisserant
parents:
diff changeset
     4
# See COPYING file for copyrights details.
8742337a9fe3 Chunk based transfer for PLC binary and extra files, and some collateral code refactoring.
Edouard Tisserant
parents:
diff changeset
     5
2492
7dd551ac2fa0 check_sources.sh makes me become even less productive
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 2487
diff changeset
     6
from __future__ import absolute_import
2463
8742337a9fe3 Chunk based transfer for PLC binary and extra files, and some collateral code refactoring.
Edouard Tisserant
parents:
diff changeset
     7
import md5
8742337a9fe3 Chunk based transfer for PLC binary and extra files, and some collateral code refactoring.
Edouard Tisserant
parents:
diff changeset
     8
2492
7dd551ac2fa0 check_sources.sh makes me become even less productive
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 2487
diff changeset
     9
2463
8742337a9fe3 Chunk based transfer for PLC binary and extra files, and some collateral code refactoring.
Edouard Tisserant
parents:
diff changeset
    10
class ConnectorBase(object):
8742337a9fe3 Chunk based transfer for PLC binary and extra files, and some collateral code refactoring.
Edouard Tisserant
parents:
diff changeset
    11
2465
47d3aea2be30 Bigger chunks, from 16k to 1M
Edouard Tisserant
parents: 2463
diff changeset
    12
    chuncksize = 1024*1024
2492
7dd551ac2fa0 check_sources.sh makes me become even less productive
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 2487
diff changeset
    13
2487
6a4f9a061994 Reworked chunk based transfer to support duplicated files (i.e. files with same content, but different names)
Edouard Tisserant
parents: 2465
diff changeset
    14
    def BlobFromFile(self, filepath, seed):
2463
8742337a9fe3 Chunk based transfer for PLC binary and extra files, and some collateral code refactoring.
Edouard Tisserant
parents:
diff changeset
    15
        s = md5.new()
2487
6a4f9a061994 Reworked chunk based transfer to support duplicated files (i.e. files with same content, but different names)
Edouard Tisserant
parents: 2465
diff changeset
    16
        s.update(seed)
6a4f9a061994 Reworked chunk based transfer to support duplicated files (i.e. files with same content, but different names)
Edouard Tisserant
parents: 2465
diff changeset
    17
        blobID = self.SeedBlob(seed)
2463
8742337a9fe3 Chunk based transfer for PLC binary and extra files, and some collateral code refactoring.
Edouard Tisserant
parents:
diff changeset
    18
        with open(filepath, "rb") as f:
2487
6a4f9a061994 Reworked chunk based transfer to support duplicated files (i.e. files with same content, but different names)
Edouard Tisserant
parents: 2465
diff changeset
    19
            while blobID == s.digest():
2492
7dd551ac2fa0 check_sources.sh makes me become even less productive
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 2487
diff changeset
    20
                chunk = f.read(self.chuncksize)
7dd551ac2fa0 check_sources.sh makes me become even less productive
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 2487
diff changeset
    21
                if len(chunk) == 0:
7dd551ac2fa0 check_sources.sh makes me become even less productive
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 2487
diff changeset
    22
                    return blobID
2463
8742337a9fe3 Chunk based transfer for PLC binary and extra files, and some collateral code refactoring.
Edouard Tisserant
parents:
diff changeset
    23
                blobID = self.AppendChunkToBlob(chunk, blobID)
8742337a9fe3 Chunk based transfer for PLC binary and extra files, and some collateral code refactoring.
Edouard Tisserant
parents:
diff changeset
    24
                s.update(chunk)