{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Timeseries\n",
    "\n",
    "`redis-py` supports [RedisTimeSeries](https://github.com/RedisTimeSeries/RedisTimeSeries/) which is a time-series-database module for Redis.\n",
    "\n",
    "This example shows how to handle timeseries data with `redis-py`."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Health check"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 1,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import redis \n",
    "\n",
    "r = redis.Redis(decode_responses=True)\n",
    "ts = r.ts()\n",
    "\n",
    "r.ping()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Simple example"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Create a timeseries"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ts.create(\"ts_key\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Add samples to the timeseries\n",
    "\n",
    "We can either set the timestamp with an UNIX timestamp in milliseconds or use * to set the timestamp based en server's clock."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "1657272304448"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ts.add(\"ts_key\", 1657265437756, 1)\n",
    "ts.add(\"ts_key\", \"1657265437757\", 2)\n",
    "ts.add(\"ts_key\", \"*\", 3)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Get the last sample"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(1657272304448, 3.0)"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ts.get(\"ts_key\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Get samples between two timestamps\n",
    "\n",
    "The minimum and maximum possible timestamps can be expressed with respectfully - and +."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[(1657265437756, 1.0), (1657265437757, 2.0), (1657272304448, 3.0)]"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ts.range(\"ts_key\", \"-\", \"+\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[(1657265437756, 1.0), (1657265437757, 2.0)]"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ts.range(\"ts_key\", 1657265437756, 1657265437757)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Delete samples between two timestamps"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Before deletion:  [(1657265437756, 1.0), (1657265437757, 2.0), (1657272304448, 3.0)]\n",
      "After deletion:   [(1657272304448, 3.0)]\n"
     ]
    }
   ],
   "source": [
    "print(\"Before deletion: \", ts.range(\"ts_key\", \"-\", \"+\"))\n",
    "ts.delete(\"ts_key\", 1657265437756, 1657265437757)\n",
    "print(\"After deletion:  \", ts.range(\"ts_key\", \"-\", \"+\"))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Multiple timeseries with labels"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ts.create(\"ts_key1\")\n",
    "ts.create(\"ts_key2\", labels={\"label1\": 1, \"label2\": 2})"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Add samples to multiple timeseries"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[1657272306147, 1657272306147]"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ts.madd([(\"ts_key1\", \"*\", 1), (\"ts_key2\", \"*\", 2)])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Add samples with labels"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "1657272306457"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ts.add(\"ts_key2\", \"*\", 2,  labels={\"label1\": 1, \"label2\": 2})\n",
    "ts.add(\"ts_key2\", \"*\", 2,  labels={\"label1\": 3, \"label2\": 4})"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Get the last sample matching specific label\n",
    "\n",
    "Get the last sample that matches \"label1=1\", see [Redis documentation](https://redis.io/commands/ts.mget/) to see the posible filter values."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[{'ts_key2': [{}, 1657272306457, 2.0]}]"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ts.mget([\"label1=1\"])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Get also the label-value pairs of the sample:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[{'ts_key2': [{'label1': '1', 'label2': '2'}, 1657272306457, 2.0]}]"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ts.mget([\"label1=1\"], with_labels=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Retention period\n",
    "\n",
    "You can specify a retention period when creating timeseries objects or when adding a sample timeseries object. Once the retention period has elapsed, the sample is removed from the timeseries."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "retention_time = 1000\n",
    "ts.create(\"ts_key_ret\", retention_msecs=retention_time)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Base timeseries:                      [(1657272307670, 1.0)]\n",
      "Timeseries after 1000 milliseconds:   [(1657272307670, 1.0)]\n"
     ]
    }
   ],
   "source": [
    "import time\n",
    "# this will be deleted in 1000 milliseconds\n",
    "ts.add(\"ts_key_ret\", \"*\", 1, retention_msecs=retention_time)\n",
    "print(\"Base timeseries:                     \", ts.range(\"ts_key_ret\", \"-\", \"+\"))\n",
    "# sleeping for 1000 milliseconds (1 second)\n",
    "time.sleep(1)\n",
    "print(\"Timeseries after 1000 milliseconds:  \", ts.range(\"ts_key_ret\", \"-\", \"+\"))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The two lists are the same, this is because the oldest values are deleted when a new sample is added."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "1657272308849"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ts.add(\"ts_key_ret\", \"*\", 10)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[(1657272308849, 10.0)]"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ts.range(\"ts_key_ret\", \"-\", \"+\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here the first sample has been deleted."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Specify duplicate policies\n",
    "\n",
    "By default, the policy for duplicates timestamp keys is set to \"BLOCK\", we cannot create two samples with the same timestamp:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "TSDB: Error at upsert, update is not supported when DUPLICATE_POLICY is set to BLOCK mode\n"
     ]
    }
   ],
   "source": [
    "ts.add(\"ts_key\", 123456789, 1)\n",
    "try:\n",
    "    ts.add(\"ts_key\", 123456789, 2)\n",
    "except Exception as err:\n",
    "    print(err)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can change this default behaviour using `duplicate_policy` parameter, for instance:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[(123456789, 2.0), (1657272304448, 3.0)]"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# using policy \"LAST\", we keep the last added sample\n",
    "ts.add(\"ts_key\", 123456789, 2, duplicate_policy=\"LAST\")\n",
    "ts.range(\"ts_key\", \"-\", \"+\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For more informations about duplicate policies, see [Redis documentation](https://redis.io/commands/ts.add/)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Using Redis TSDB to keep track of a value"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "1657272310241"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ts.add(\"ts_key_incr\", \"*\", 0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Increment the value:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [],
   "source": [
    "for _ in range(10):\n",
    "    ts.incrby(\"ts_key_incr\", 1)\n",
    "    # sleeping a bit so the timestamp are not duplicates\n",
    "    time.sleep(0.01)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[(1657272310241, 0.0),\n",
       " (1657272310533, 1.0),\n",
       " (1657272310545, 2.0),\n",
       " (1657272310556, 3.0),\n",
       " (1657272310567, 4.0),\n",
       " (1657272310578, 5.0),\n",
       " (1657272310589, 6.0),\n",
       " (1657272310600, 7.0),\n",
       " (1657272310611, 8.0),\n",
       " (1657272310622, 9.0),\n",
       " (1657272310632, 10.0)]"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ts.range(\"ts_key_incr\", \"-\", \"+\")"
   ]
  },
  {
   "cell_type": "markdown",
   "source": [
    "## How to execute multi-key commands on Open Source Redis Cluster"
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "outputs": [
    {
     "data": {
      "text/plain": "[{'ts_key1': [{}, 1670927124746, 2.0]}, {'ts_key2': [{}, 1670927124748, 10.0]}]"
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import redis\n",
    "\n",
    "r = redis.RedisCluster(host=\"localhost\", port=46379)\n",
    "\n",
    "# This command should be executed on all cluster nodes after creation and any re-sharding\n",
    "# Please note that this command is internal and will be deprecated in the future\n",
    "r.execute_command(\"timeseries.REFRESHCLUSTER\", target_nodes=\"primaries\")\n",
    "\n",
    "# Now multi-key commands can be executed\n",
    "ts = r.ts()\n",
    "ts.add(\"ts_key1\", \"*\", 2,  labels={\"label1\": 1, \"label2\": 2})\n",
    "ts.add(\"ts_key2\", \"*\", 10,  labels={\"label1\": 1, \"label2\": 2})\n",
    "ts.mget([\"label1=1\"])"
   ],
   "metadata": {
    "collapsed": false
   }
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3.9.2 64-bit",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.2"
  },
  "orig_nbformat": 4,
  "vscode": {
   "interpreter": {
    "hash": "916dbcbb3f70747c44a77c7bcd40155683ae19c65e1c03b4aa3499c5328201f1"
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}