{"id":183,"date":"2014-12-30T17:38:44","date_gmt":"2014-12-30T17:38:44","guid":{"rendered":"http:\/\/www.clowersresearch.com\/main\/?p=183"},"modified":"2016-06-14T14:37:07","modified_gmt":"2016-06-14T14:37:07","slug":"savitzky-golay-smoothing-gui","status":"publish","type":"post","link":"http:\/\/www.clowersresearch.com\/main\/savitzky-golay-smoothing-gui\/","title":{"rendered":"Savitzky-Golay Smoothing GUI"},"content":{"rendered":"

\"Simple<\/p>\n

In an effort to create a set of simple tools that are useful for data processing and realtime analysis of data we’ve been exploring a range of tools. \u00a0Granted there are a number of canned solutions in existence (e.g. National Instruments), however, to avoid the long-term challenges of compatibility we are looking for tools that can better serve our research goals. \u00a0Two packages that we’ve began to lean more heavily upon include pyqtgraph<\/a> and guidata<\/a>. \u00a0Both use PyQt4 and are compatible with Pyside for GUI rendering and construction. \u00a0Matplotlib is quite mature but it has been our experience that pyqtgraph is quite a bit faster for plotting data in realtime.<\/p>\n

The code below integrates pyqtgraph directly into the guidata framework. \u00a0This is not a huge stretch as the pyqtgraph widgets integrate directly with the QWidget class in PyQt4. \u00a0For those looking for an example the following code illustrate very simply how to integrate one of these plots and update it using simulated data along with the ability to alter the smoothing parameters of the raw data on the fly. \u00a0One might envision the use of this approach to capture data from a streaming device (more on that later).\u00a0It should be noted that the file loading feature has been disabled but it would’t be a huge stretch to re-enable this functionality for single spectra.<\/p>\n


\n<\/a><\/p>\n

\r\n# -*- coding: utf-8 -*-\r\n# Adapted from guidata examples:\r\n# Copyright \u00a9 2009-2010 CEA\r\n# Pierre Raybaut\r\n# Licensed under the terms of the CECILL License\r\n# (see guidata\/__init__.py for details)\r\n# Adapted by Brian Clowers brian.clowers@wsu.edu\r\n\r\n"""\r\nDataSetEditGroupBox and DataSetShowGroupBox demo\r\n\r\nThese group box widgets are intended to be integrated in a GUI application\r\nlayout, showing read-only parameter sets or allowing to edit parameter values.\r\n"""\r\n\r\nSHOW = True # Show test in GUI-based test launcher\r\n\r\nimport tempfile, atexit, shutil, datetime, numpy as N\r\n\r\nfrom guidata.qt.QtGui import QMainWindow, QSplitter\r\nfrom guidata.qt.QtCore import SIGNAL, QTimer\r\nfrom guidata.qt import QtCore\r\n\r\nfrom guidata.dataset.datatypes import (DataSet, BeginGroup, EndGroup, BeginTabGroup, EndTabGroup)\r\nfrom guidata.dataset.dataitems import (FloatItem, IntItem, BoolItem, ChoiceItem, MultipleChoiceItem, ImageChoiceItem, FilesOpenItem, StringItem, TextItem, ColorItem, FileSaveItem, FileOpenItem, DirectoryItem, FloatArrayItem, DateItem, DateTimeItem)\r\nfrom guidata.dataset.qtwidgets import DataSetShowGroupBox, DataSetEditGroupBox\r\nfrom guidata.configtools import get_icon\r\nfrom guidata.qthelpers import create_action, add_actions, get_std_icon\r\n\r\n# Local test import:\r\nfrom guidata.tests.activable_dataset import ExampleDataSet\r\n\r\nimport sys, os\r\nimport pyqtgraph as PG\r\n\r\n#-----------------------------------\r\ndef simpleSmooth(fileName, polyOrder, pointLength, plotSmoothed = False, saveSmoothed = True):\r\n    if not os.path.isfile(fileName):\r\n        return False\r\n    rawArray = get_ascii_data(fileName)\r\n    #savitzky_golay(data, kernel = 11, order = 4)\r\n    smoothArray = savitzky_golay(rawArray, kernel = pointLength, order = polyOrder)\r\n    if plotSmoothed:\r\n        plot_smoothed(smoothArray, rawArray, True)\r\n\r\n    if saveSmoothed:\r\n        newFileName = fileName.split(".")[0]\r\n        newFileName+="_smth.csv"\r\n \r\n    N.savetxt(newFileName, smoothArray, delimiter = ',', fmt = '%.4f')\r\n\r\n    return smoothArray\r\n\r\n#-----------------------------------\r\n\r\ndef get_ascii_data(filename):\r\n    data_spectrum=N.loadtxt(filename,delimiter = ',', skiprows=0)##remember to change this depending on file format\r\n    return data_spectrum\r\n\r\n#-----------------------------------\r\ndef savitzky_golay(data, kernel = 11, order = 4):\r\n """\r\n applies a Savitzky-Golay filter\r\n input parameters:\r\n - data => data as a 1D numpy array\r\n - kernel => a positive integer > 2*order giving the kernel size\r\n - order => order of the polynomal\r\n returns smoothed data as a numpy array\r\n\r\n invoke like:\r\n smoothed = savitzky_golay(<rough>, [kernel = value], [order = value]\r\n\r\n From scipy website\r\n """\r\n try:\r\n kernel = abs(int(kernel))\r\n order = abs(int(order))\r\n except ValueError, msg:\r\n raise ValueError("kernel and order have to be of type int (floats will be converted).")\r\n if kernel % 2 != 1 or kernel < 1:\r\n raise TypeError("kernel size must be a positive odd number, was: %d" % kernel)\r\n if kernel < order + 2:\r\n raise TypeError("kernel is to small for the polynomals\\nshould be > order + 2")\r\n\r\n # a second order polynomal has 3 coefficients\r\n order_range = range(order+1)\r\n half_window = (kernel -1) \/\/ 2\r\n b = N.mat([[k**i for i in order_range] for k in range(-half_window, half_window+1)])\r\n # since we don't want the derivative, else choose [1] or [2], respectively\r\n m = N.linalg.pinv(b).A[0]\r\n window_size = len(m)\r\n half_window = (window_size-1) \/\/ 2\r\n\r\n # precompute the offset values for better performance\r\n offsets = range(-half_window, half_window+1)\r\n offset_data = zip(offsets, m)\r\n\r\n smooth_data = list()\r\n\r\n # temporary data, with padded zeros (since we want the same length after smoothing)\r\n #data = numpy.concatenate((numpy.zeros(half_window), data, numpy.zeros(half_window)))\r\n # temporary data, with padded first\/last values (since we want the same length after smoothing)\r\n firstval=data[0]\r\n lastval=data[len(data)-1]\r\n data = N.concatenate((N.zeros(half_window)+firstval, data, N.zeros(half_window)+lastval))\r\n\r\n for i in range(half_window, len(data) - half_window):\r\n value = 0.0\r\n for offset, weight in offset_data:\r\n value += weight * data[i + offset]\r\n smooth_data.append(value)\r\n return N.array(smooth_data)\r\n\r\n#-----------------------------------\r\n\r\ndef first_derivative(y_data):\r\n """\\\r\n calculates the derivative\r\n """\r\n \r\n y = (y_data[1:]-y_data[:-1])\r\n \r\n dy = y\/2#((x_data[1:]-x_data[:-1])\/2)\r\n\r\n return dy\r\n\r\n#-----------------------------------\r\nclass SmoothGUI(DataSet):\r\n """\r\n Simple Smoother\r\n A simple application for smoothing a 1D text file at this stage. \r\n Follows the KISS principle.\r\n """\r\n fname = FileOpenItem("Open file", ("txt", "csv"), "")\r\n\r\n kernel = FloatItem("Smooth Point Length", default=7, min=1, max=101, step=2, slider=True) \r\n order = IntItem("Polynomial Order", default=3, min=3, max=17, slider=True)\r\n saveBool = BoolItem("Save Plot Output", default = True)\r\n plotBool = BoolItem("Plot Smoothed", default = True).set_pos(col=1)\r\n #color = ColorItem("Color", default="red")\r\n \r\n#-----------------------------------\r\nclass MainWindow(QMainWindow):\r\n def __init__(self):\r\n QMainWindow.__init__(self)\r\n self.setWindowIcon(get_icon('python.png'))\r\n self.setWindowTitle("Simple Smoother")\r\n \r\n # Instantiate dataset-related widgets:\r\n self.smoothGB = DataSetEditGroupBox("Smooth Parameters",\r\n SmoothGUI, comment='')\r\n\r\n self.connect(self.smoothGB, SIGNAL("apply_button_clicked()"),\r\n self.update_window)\r\n\r\n self.fileName = ''\r\n\r\n self.kernel = 15\r\n self.order = 3\r\n self.pw = PG.PlotWidget(name='Plot1')\r\n self.pw.showGrid(x=True, y = True)\r\n\r\n self.p1 = self.pw.plot()\r\n self.p1.setPen('g', alpha = 1.0)#Does alpha even do anything?\r\n self.p2 = self.pw.plot(pen = 'y')\r\n self.pw.setLabel('left', 'Value', units='V')\r\n self.pw.setLabel('bottom', 'Time', units='s')\r\n\r\n splitter = QSplitter(QtCore.Qt.Vertical, parent = self)\r\n\r\n splitter.addWidget(self.smoothGB)\r\n splitter.addWidget(self.pw)\r\n self.setCentralWidget(splitter)\r\n self.setContentsMargins(10, 5, 10, 5)\r\n \r\n # File menu\r\n file_menu = self.menuBar().addMenu("File")\r\n quit_action = create_action(self, "Quit",\r\n shortcut="Ctrl+Q",\r\n icon=get_std_icon("DialogCloseButton"),\r\n tip="Quit application",\r\n triggered=self.close)\r\n add_actions(file_menu, (quit_action, ))\r\n \r\n ## Start a timer to rapidly update the plot in pw\r\n self.t = QTimer()\r\n self.t.timeout.connect(self.updateData)\r\n self.t.start(1000) \r\n\r\n def rand(self,n):\r\n data = N.random.random(n)\r\n data[int(n*0.1):int(n*0.23)] += .5\r\n data[int(n*0.18):int(n*0.25)] += 1\r\n data[int(n*0.1):int(n*0.13)] *= 2.5\r\n data[int(n*0.18)] *= 2\r\n data *= 1e-12\r\n return data, N.arange(n, n+len(data)) \/ float(n)\r\n \r\n\r\n def updateData(self):\r\n yd, xd = self.rand(100)\r\n ydSmooth = savitzky_golay(yd, kernel = self.kernel, order = self.order)\r\n \r\n if self.smoothGB.dataset.plotBool:\r\n self.p2.setData(y=ydSmooth, x = xd, clear = True)\r\n self.p1.setData(y=yd*-1, x=xd, clear = True)\r\n else:\r\n self.p1.setData(y=yd, x=xd, clear = True)\r\n self.p2.setData(y=[yd[0]], x = [xd[0]], clear = True)\r\n\r\n if self.smoothGB.dataset.saveBool:\r\n if os.path.isfile(self.fileName):\r\n newFileName = self.fileName.split(".")[0]\r\n \r\n else:\r\n newFileName = "test"\r\n newFileName+="_smth.csv"\r\n \r\n N.savetxt(newFileName, ydSmooth, delimiter = ',')#, fmt = '%.4f')\r\n\r\n\r\n \r\n def update_window(self):\r\n dataset = self.smoothGB.dataset\r\n self.order = dataset.order\r\n self.kernel = dataset.kernel\r\n self.fileName = dataset.fname\r\n\r\n \r\n \r\nif __name__ == '__main__':\r\n from guidata.qt.QtGui import QApplication\r\n app = QApplication(sys.argv)\r\n window = MainWindow()\r\n window.show()\r\n sys.exit(app.exec_())\r\n<\/pre>\n","protected":false},"excerpt":{"rendered":"
In an effort to create a set of simple tools that are useful for data processing and realtime analysis of data we’ve been exploring a range of tools. \u00a0Granted there are a number of canned solutions in existence (e.g. National Instruments), however, to avoid the long-term challenges of compatibility we are looking for tools that […]<\/p>\n","protected":false},"author":1,"featured_media":184,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"spay_email":""},"categories":[6,11],"tags":[16,15,14,4,20],"jetpack_featured_media_url":"http:\/\/www.clowersresearch.com\/main\/wp-content\/uploads\/2014\/12\/Simple_Smoother.png","_links":{"self":[{"href":"http:\/\/www.clowersresearch.com\/main\/wp-json\/wp\/v2\/posts\/183"}],"collection":[{"href":"http:\/\/www.clowersresearch.com\/main\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"http:\/\/www.clowersresearch.com\/main\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"http:\/\/www.clowersresearch.com\/main\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"http:\/\/www.clowersresearch.com\/main\/wp-json\/wp\/v2\/comments?post=183"}],"version-history":[{"count":4,"href":"http:\/\/www.clowersresearch.com\/main\/wp-json\/wp\/v2\/posts\/183\/revisions"}],"predecessor-version":[{"id":281,"href":"http:\/\/www.clowersresearch.com\/main\/wp-json\/wp\/v2\/posts\/183\/revisions\/281"}],"wp:featuredmedia":[{"embeddable":true,"href":"http:\/\/www.clowersresearch.com\/main\/wp-json\/wp\/v2\/media\/184"}],"wp:attachment":[{"href":"http:\/\/www.clowersresearch.com\/main\/wp-json\/wp\/v2\/media?parent=183"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"http:\/\/www.clowersresearch.com\/main\/wp-json\/wp\/v2\/categories?post=183"},{"taxonomy":"post_tag","embeddable":true,"href":"http:\/\/www.clowersresearch.com\/main\/wp-json\/wp\/v2\/tags?post=183"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}