author | Edouard Tisserant |
Mon, 14 Jun 2021 16:48:39 +0200 | |
branch | svghmi |
changeset 3257 | 095c73591b7e |
child 2742 | 5f7445b582d4 |
permissions | -rw-r--r-- |
3257
095c73591b7e
IDE: Cleaned up some useless tests in variable trace data handling code, changed from bare numpy arrays to RingBuffers inorder to avoid RAM outage and crash after long tracing session.
Edouard Tisserant
parents:
diff
changeset
|
1 |
#!/usr/bin/env python |
095c73591b7e
IDE: Cleaned up some useless tests in variable trace data handling code, changed from bare numpy arrays to RingBuffers inorder to avoid RAM outage and crash after long tracing session.
Edouard Tisserant
parents:
diff
changeset
|
2 |
# -*- coding: utf-8 -*- |
095c73591b7e
IDE: Cleaned up some useless tests in variable trace data handling code, changed from bare numpy arrays to RingBuffers inorder to avoid RAM outage and crash after long tracing session.
Edouard Tisserant
parents:
diff
changeset
|
3 |
|
095c73591b7e
IDE: Cleaned up some useless tests in variable trace data handling code, changed from bare numpy arrays to RingBuffers inorder to avoid RAM outage and crash after long tracing session.
Edouard Tisserant
parents:
diff
changeset
|
4 |
# This file is part of Beremiz |
095c73591b7e
IDE: Cleaned up some useless tests in variable trace data handling code, changed from bare numpy arrays to RingBuffers inorder to avoid RAM outage and crash after long tracing session.
Edouard Tisserant
parents:
diff
changeset
|
5 |
# Copyright (C) 2021: Edouard TISSERANT |
095c73591b7e
IDE: Cleaned up some useless tests in variable trace data handling code, changed from bare numpy arrays to RingBuffers inorder to avoid RAM outage and crash after long tracing session.
Edouard Tisserant
parents:
diff
changeset
|
6 |
# |
095c73591b7e
IDE: Cleaned up some useless tests in variable trace data handling code, changed from bare numpy arrays to RingBuffers inorder to avoid RAM outage and crash after long tracing session.
Edouard Tisserant
parents:
diff
changeset
|
7 |
# See COPYING file for copyrights details. |
095c73591b7e
IDE: Cleaned up some useless tests in variable trace data handling code, changed from bare numpy arrays to RingBuffers inorder to avoid RAM outage and crash after long tracing session.
Edouard Tisserant
parents:
diff
changeset
|
8 |
|
095c73591b7e
IDE: Cleaned up some useless tests in variable trace data handling code, changed from bare numpy arrays to RingBuffers inorder to avoid RAM outage and crash after long tracing session.
Edouard Tisserant
parents:
diff
changeset
|
9 |
# Based on Eelco Hoogendoorn stackoverflow answer about RingBuffer with numpy |
095c73591b7e
IDE: Cleaned up some useless tests in variable trace data handling code, changed from bare numpy arrays to RingBuffers inorder to avoid RAM outage and crash after long tracing session.
Edouard Tisserant
parents:
diff
changeset
|
10 |
|
095c73591b7e
IDE: Cleaned up some useless tests in variable trace data handling code, changed from bare numpy arrays to RingBuffers inorder to avoid RAM outage and crash after long tracing session.
Edouard Tisserant
parents:
diff
changeset
|
11 |
import numpy as np |
095c73591b7e
IDE: Cleaned up some useless tests in variable trace data handling code, changed from bare numpy arrays to RingBuffers inorder to avoid RAM outage and crash after long tracing session.
Edouard Tisserant
parents:
diff
changeset
|
12 |
|
095c73591b7e
IDE: Cleaned up some useless tests in variable trace data handling code, changed from bare numpy arrays to RingBuffers inorder to avoid RAM outage and crash after long tracing session.
Edouard Tisserant
parents:
diff
changeset
|
13 |
|
095c73591b7e
IDE: Cleaned up some useless tests in variable trace data handling code, changed from bare numpy arrays to RingBuffers inorder to avoid RAM outage and crash after long tracing session.
Edouard Tisserant
parents:
diff
changeset
|
14 |
class RingBuffer(object): |
095c73591b7e
IDE: Cleaned up some useless tests in variable trace data handling code, changed from bare numpy arrays to RingBuffers inorder to avoid RAM outage and crash after long tracing session.
Edouard Tisserant
parents:
diff
changeset
|
15 |
def __init__(self, width=None, size=65536, padding=None): |
095c73591b7e
IDE: Cleaned up some useless tests in variable trace data handling code, changed from bare numpy arrays to RingBuffers inorder to avoid RAM outage and crash after long tracing session.
Edouard Tisserant
parents:
diff
changeset
|
16 |
self.size = size |
095c73591b7e
IDE: Cleaned up some useless tests in variable trace data handling code, changed from bare numpy arrays to RingBuffers inorder to avoid RAM outage and crash after long tracing session.
Edouard Tisserant
parents:
diff
changeset
|
17 |
self.padding = size if padding is None else padding |
095c73591b7e
IDE: Cleaned up some useless tests in variable trace data handling code, changed from bare numpy arrays to RingBuffers inorder to avoid RAM outage and crash after long tracing session.
Edouard Tisserant
parents:
diff
changeset
|
18 |
shape = (self.size+self.padding,) |
095c73591b7e
IDE: Cleaned up some useless tests in variable trace data handling code, changed from bare numpy arrays to RingBuffers inorder to avoid RAM outage and crash after long tracing session.
Edouard Tisserant
parents:
diff
changeset
|
19 |
if width : |
095c73591b7e
IDE: Cleaned up some useless tests in variable trace data handling code, changed from bare numpy arrays to RingBuffers inorder to avoid RAM outage and crash after long tracing session.
Edouard Tisserant
parents:
diff
changeset
|
20 |
shape += (width,) |
095c73591b7e
IDE: Cleaned up some useless tests in variable trace data handling code, changed from bare numpy arrays to RingBuffers inorder to avoid RAM outage and crash after long tracing session.
Edouard Tisserant
parents:
diff
changeset
|
21 |
self.buffer = np.zeros(shape) |
095c73591b7e
IDE: Cleaned up some useless tests in variable trace data handling code, changed from bare numpy arrays to RingBuffers inorder to avoid RAM outage and crash after long tracing session.
Edouard Tisserant
parents:
diff
changeset
|
22 |
self.counter = 0 |
095c73591b7e
IDE: Cleaned up some useless tests in variable trace data handling code, changed from bare numpy arrays to RingBuffers inorder to avoid RAM outage and crash after long tracing session.
Edouard Tisserant
parents:
diff
changeset
|
23 |
self.full = False |
095c73591b7e
IDE: Cleaned up some useless tests in variable trace data handling code, changed from bare numpy arrays to RingBuffers inorder to avoid RAM outage and crash after long tracing session.
Edouard Tisserant
parents:
diff
changeset
|
24 |
|
095c73591b7e
IDE: Cleaned up some useless tests in variable trace data handling code, changed from bare numpy arrays to RingBuffers inorder to avoid RAM outage and crash after long tracing session.
Edouard Tisserant
parents:
diff
changeset
|
25 |
def append(self, data): |
095c73591b7e
IDE: Cleaned up some useless tests in variable trace data handling code, changed from bare numpy arrays to RingBuffers inorder to avoid RAM outage and crash after long tracing session.
Edouard Tisserant
parents:
diff
changeset
|
26 |
"""this is an O(n) operation""" |
095c73591b7e
IDE: Cleaned up some useless tests in variable trace data handling code, changed from bare numpy arrays to RingBuffers inorder to avoid RAM outage and crash after long tracing session.
Edouard Tisserant
parents:
diff
changeset
|
27 |
data = data[-self.padding:] |
095c73591b7e
IDE: Cleaned up some useless tests in variable trace data handling code, changed from bare numpy arrays to RingBuffers inorder to avoid RAM outage and crash after long tracing session.
Edouard Tisserant
parents:
diff
changeset
|
28 |
n = len(data) |
095c73591b7e
IDE: Cleaned up some useless tests in variable trace data handling code, changed from bare numpy arrays to RingBuffers inorder to avoid RAM outage and crash after long tracing session.
Edouard Tisserant
parents:
diff
changeset
|
29 |
if self.remaining < n: self.compact() |
095c73591b7e
IDE: Cleaned up some useless tests in variable trace data handling code, changed from bare numpy arrays to RingBuffers inorder to avoid RAM outage and crash after long tracing session.
Edouard Tisserant
parents:
diff
changeset
|
30 |
self.buffer[self.counter+self.size:][:n] = data |
095c73591b7e
IDE: Cleaned up some useless tests in variable trace data handling code, changed from bare numpy arrays to RingBuffers inorder to avoid RAM outage and crash after long tracing session.
Edouard Tisserant
parents:
diff
changeset
|
31 |
self.counter += n |
095c73591b7e
IDE: Cleaned up some useless tests in variable trace data handling code, changed from bare numpy arrays to RingBuffers inorder to avoid RAM outage and crash after long tracing session.
Edouard Tisserant
parents:
diff
changeset
|
32 |
|
095c73591b7e
IDE: Cleaned up some useless tests in variable trace data handling code, changed from bare numpy arrays to RingBuffers inorder to avoid RAM outage and crash after long tracing session.
Edouard Tisserant
parents:
diff
changeset
|
33 |
@property |
095c73591b7e
IDE: Cleaned up some useless tests in variable trace data handling code, changed from bare numpy arrays to RingBuffers inorder to avoid RAM outage and crash after long tracing session.
Edouard Tisserant
parents:
diff
changeset
|
34 |
def count(self): |
095c73591b7e
IDE: Cleaned up some useless tests in variable trace data handling code, changed from bare numpy arrays to RingBuffers inorder to avoid RAM outage and crash after long tracing session.
Edouard Tisserant
parents:
diff
changeset
|
35 |
return self.counter if not self.full else self.size |
095c73591b7e
IDE: Cleaned up some useless tests in variable trace data handling code, changed from bare numpy arrays to RingBuffers inorder to avoid RAM outage and crash after long tracing session.
Edouard Tisserant
parents:
diff
changeset
|
36 |
|
095c73591b7e
IDE: Cleaned up some useless tests in variable trace data handling code, changed from bare numpy arrays to RingBuffers inorder to avoid RAM outage and crash after long tracing session.
Edouard Tisserant
parents:
diff
changeset
|
37 |
@property |
095c73591b7e
IDE: Cleaned up some useless tests in variable trace data handling code, changed from bare numpy arrays to RingBuffers inorder to avoid RAM outage and crash after long tracing session.
Edouard Tisserant
parents:
diff
changeset
|
38 |
def remaining(self): |
095c73591b7e
IDE: Cleaned up some useless tests in variable trace data handling code, changed from bare numpy arrays to RingBuffers inorder to avoid RAM outage and crash after long tracing session.
Edouard Tisserant
parents:
diff
changeset
|
39 |
return self.padding-self.counter |
095c73591b7e
IDE: Cleaned up some useless tests in variable trace data handling code, changed from bare numpy arrays to RingBuffers inorder to avoid RAM outage and crash after long tracing session.
Edouard Tisserant
parents:
diff
changeset
|
40 |
|
095c73591b7e
IDE: Cleaned up some useless tests in variable trace data handling code, changed from bare numpy arrays to RingBuffers inorder to avoid RAM outage and crash after long tracing session.
Edouard Tisserant
parents:
diff
changeset
|
41 |
@property |
095c73591b7e
IDE: Cleaned up some useless tests in variable trace data handling code, changed from bare numpy arrays to RingBuffers inorder to avoid RAM outage and crash after long tracing session.
Edouard Tisserant
parents:
diff
changeset
|
42 |
def view(self): |
095c73591b7e
IDE: Cleaned up some useless tests in variable trace data handling code, changed from bare numpy arrays to RingBuffers inorder to avoid RAM outage and crash after long tracing session.
Edouard Tisserant
parents:
diff
changeset
|
43 |
"""this is always an O(1) operation""" |
095c73591b7e
IDE: Cleaned up some useless tests in variable trace data handling code, changed from bare numpy arrays to RingBuffers inorder to avoid RAM outage and crash after long tracing session.
Edouard Tisserant
parents:
diff
changeset
|
44 |
return self.buffer[self.counter:][:self.size] |
095c73591b7e
IDE: Cleaned up some useless tests in variable trace data handling code, changed from bare numpy arrays to RingBuffers inorder to avoid RAM outage and crash after long tracing session.
Edouard Tisserant
parents:
diff
changeset
|
45 |
|
095c73591b7e
IDE: Cleaned up some useless tests in variable trace data handling code, changed from bare numpy arrays to RingBuffers inorder to avoid RAM outage and crash after long tracing session.
Edouard Tisserant
parents:
diff
changeset
|
46 |
def compact(self): |
095c73591b7e
IDE: Cleaned up some useless tests in variable trace data handling code, changed from bare numpy arrays to RingBuffers inorder to avoid RAM outage and crash after long tracing session.
Edouard Tisserant
parents:
diff
changeset
|
47 |
""" |
095c73591b7e
IDE: Cleaned up some useless tests in variable trace data handling code, changed from bare numpy arrays to RingBuffers inorder to avoid RAM outage and crash after long tracing session.
Edouard Tisserant
parents:
diff
changeset
|
48 |
note: only when this function is called, is an O(size) performance hit incurred, |
095c73591b7e
IDE: Cleaned up some useless tests in variable trace data handling code, changed from bare numpy arrays to RingBuffers inorder to avoid RAM outage and crash after long tracing session.
Edouard Tisserant
parents:
diff
changeset
|
49 |
and this cost is amortized over the whole padding space |
095c73591b7e
IDE: Cleaned up some useless tests in variable trace data handling code, changed from bare numpy arrays to RingBuffers inorder to avoid RAM outage and crash after long tracing session.
Edouard Tisserant
parents:
diff
changeset
|
50 |
""" |
095c73591b7e
IDE: Cleaned up some useless tests in variable trace data handling code, changed from bare numpy arrays to RingBuffers inorder to avoid RAM outage and crash after long tracing session.
Edouard Tisserant
parents:
diff
changeset
|
51 |
print 'compacting' |
095c73591b7e
IDE: Cleaned up some useless tests in variable trace data handling code, changed from bare numpy arrays to RingBuffers inorder to avoid RAM outage and crash after long tracing session.
Edouard Tisserant
parents:
diff
changeset
|
52 |
self.buffer[:self.size] = self.view |
095c73591b7e
IDE: Cleaned up some useless tests in variable trace data handling code, changed from bare numpy arrays to RingBuffers inorder to avoid RAM outage and crash after long tracing session.
Edouard Tisserant
parents:
diff
changeset
|
53 |
self.counter = 0 |
095c73591b7e
IDE: Cleaned up some useless tests in variable trace data handling code, changed from bare numpy arrays to RingBuffers inorder to avoid RAM outage and crash after long tracing session.
Edouard Tisserant
parents:
diff
changeset
|
54 |
self.full = True |
095c73591b7e
IDE: Cleaned up some useless tests in variable trace data handling code, changed from bare numpy arrays to RingBuffers inorder to avoid RAM outage and crash after long tracing session.
Edouard Tisserant
parents:
diff
changeset
|
55 |