redis + python: web cache management | as a usage (pattern

I needed a simple cache solution using with flask and therefore used SimpleCache of werkzeug. I used to cache some json data returning from inner modules such as list of things, list of that, list of this; using a pattern similar to this one:

[Customer1.UserList.Page1] = {1:'bla', 2:'foo', 3:'bar'}
[Customer1.UserList.Page2] = {4:'bla', 5:'foo', 6:'bar'}
[Customer2.UserList.Page2] = {34:'bla', 35:'foo', 36:'bar'}

With a managable and smart expiry timeout (ie. 20 secs), this worked nice.

There were 3 problems in front of me, [1] on each action, I needed check if result is on cache, if not evaluate the action then push to cache, then code comes messy; so it should have had simpler usage [2] I needed to extend the timeout so that application is less busier overall, and [3] whenever a user makes a data manipulation, I needed to remove the keys with some specific pattern for the user.

Looking around, I found nearly nothing in terms of w/pattern key deleting.

At least, I found the LUA script sheerun shared on stackoverflow. Then wrote a method wrapper which would check the cache for me and if nothing there, evaluate the result and push it into the cache. So this section would be “more” automized.

Pushing to the cache…

This is the wrapper:

def getFromCacheOrSet(callable, *params, **opts):
    key = opts.get('key', callable.__name__) + '.' + '.'.join(map(unicode, params)).encode('utf-8')
    to = opts.get('timeout', 30)
    extendTime = opts.get('extendTime', False)
        v = __cached.get(key)
        if v is None:
            v = callable(*params)
            __cached.setex(key, to, v)
            return v
            if extendTime: __cached.expire(key, to)
            return v    #fromCache
    except Exception as e:
        print('Error: get from cache fai

What it makes is that, if you would like to run a multiplication method such as multiply(4,16) you pass the method, its parameters to the wrapper so that it takes care of getting from the cache or evaluating. The point is wierdest line is obviously the “key” section which makes itself a key for the cache. Key is builded from the name of the method and its argument values.

For multiply(4, 16) key becomes “multiply.4.16″, if you rather would like to choose another string for this operation, you can pass it as key=”MultiplicationOp” in opts as in:

getFromCacheOrSet(multiply, 4, 16, key="MultiplicationOp", timeout=20)    # 20 secs

then here we get this key on the cache:

MultiplicationOp.4.16 = 64

It evaluates and pushes this information to the cache.

About pattern deleting…

This is not the best way by far. However, since I could not find a better way, it does the job that I have to settle. As I mentioned earlier, thanks to sheerun@stackoverflow I have something.

This is the removing part:

def removeFromCache(callable, *params, **opts):
        key = opts.get('key', callable.__name__) + '.' + '.'.join(map(unicode, params)).encode('utf-8')
        __cached_del_keys(keys=[], args=[key + '*'])
    except Exception as e:
        print('Error: Couldnot delete from cache. Hopefully will expire itself soon.', e)
        return False

What happens here is, deleting the matching keys fits the call with as we previously made for multiplication. Such as:

removeFromCache(multiply, 4, key="MultiplicationOp")

key value is mandatory only if you used it while setting the value (obviously). Warning should be this, in here there is no 2nd argument which get to be 16 here. This is because we would like to delete all “multiply.4.*” keys.

The LUA script which I call before the methods above:

__cached = redis.StrictRedis(host='localhost', port=6379, db=0) ##### db1 !
__DEL_LUA = ("""local keys ='keys', ARGV[1])
 for i=1,#keys,5000 do'del', unpack(keys, i, math.min(i+4999, #keys)))
 return keys""")
__cached_del_keys = __cached.register_script(__DEL_LUA)

Overall usage, eg:

getFromCacheOrSet(getCustomerList, "customer_1", "arg1", "arg2", 3, timeout=1200)  # 1200sec = 20 mins expire
getFromCacheOrSet(getCustomerList, "customer_1", "arg1", "arg2", 2, timeout=1200)
getFromCacheOrSet(getCustomerList, "customer_2", "arg5", "arg6", 1, timeout=1200)
## customer_1 does something to list, have to refresh
removeFromCache(getCustomerList, "customer_1", "arg1")  # remove all getCustomerList.customer_1.arg1.***

I hope anyone would be happy to use.

As a note; it’s better serializing the objects before pushing into the cache and deserializing respective text into object after getting from cache server. These might help:

def serialize(obj):
    return base64.b64encode(pickle.dumps(obj))
def deserialize(s):
    return pickle.loads(base64.b64decode(s))

1. redis-py
2. redis
3. werkzeug SimpleCache and RedisCache
4. flask
5. issue on stackoverflow

Use your PuTTY SSH Key on MacOs

I have had the problem authentication with my ssh server with a private key. It asks password despite no password needed. Here is the log:

mememe@Mac:~$ ssh -v -i ~/.ssh/id.ppk
OpenSSH_6.2p2, OSSLShim 0.9.8r 8 Dec 2011
debug1: Reading configuration data /Users/<username>/.ssh/config
debug1: Reading configuration data /etc/ssh_config
debug1: /etc/ssh_config line 20: Applying options for *
debug1: /etc/ssh_config line 102: Applying options for *
debug1: Connecting [999.1.1.1] port 22.
debug1: Connection established.
debug1: identity file /Users/<username>/.ssh/id.ppk type -1
debug1: identity file /Users/<username>/<username>/id.ppk-cert type -1
debug1: Enabling compatibility mode for protocol 2.0
debug1: Local version string SSH-2.0-OpenSSH_6.2
debug1: Remote protocol version 2.0, remote software version OpenSSH_6.6.1p1 Ubuntu-2ubuntu2
debug1: match: OpenSSH_6.6.1p1 Ubuntu-2ubuntu2 pat OpenSSH*
debug1: SSH2_MSG_KEXINIT sent
debug1: SSH2_MSG_KEXINIT received
debug1: kex: server->client aes128-ctr none
debug1: kex: client->server aes128-ctr none
debug1: SSH2_MSG_KEX_DH_GEX_REQUEST(1024<1024<8192) sent
debug1: expecting SSH2_MSG_KEX_DH_GEX_GROUP
debug1: SSH2_MSG_KEX_DH_GEX_INIT sent
debug1: expecting SSH2_MSG_KEX_DH_GEX_REPLY
Host sfs
debug1: Server host key: RSA 43:xx:72:xx:xx:b0:xx:8a:xx:xx:xx:xx:xx:xx:a7
debug1: Host '' is known and matches the RSA host key.
debug1: Found key in /Users/<username>/.ssh/known_hosts:5
debug1: ssh_rsa_verify: signature correct
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug1: SSH2_MSG_NEWKEYS received
debug1: Roaming not allowed by server
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug1: Authentications that can continue: publickey
debug1: Next authentication method: publickey
debug1: Trying private key: /Users/<username>/.ssh/id.ppk
debug1: key_parse_private_pem: PEM_read_PrivateKey failed
debug1: read PEM private key done: type <unknown>
debug1: No more authentication methods to try.
Permission denied (publickey).

So it appears that, I tried to use PuTTY gen generated key file with OpenSSH. Therefore, without windows I had to use the steps:

1) This installs putty along with puttygen: (MacPorts needed)

$ sudo port install putty

2) This converts my ssh keygen into OpenSSH key type:

$ puttygen id.ppk -O private-openssh -o id_openssh 

now done. let’s go:

$ ssh -i ~/.ssh/id_openssh

goes ok for me.

Simple Pair Programming

There are nice samples for pair programming solutions. Well, I have seen a few, too.

I needed a simpler and quicker solution so here I went like this:

add to end of /etc/bash.bashrc:

exit() {
    if [[ -z $TMUX ]]; then
       builtin exit
       tmux detach

close() {
    builtin exit

if [[ "x" == "x$TMUX" ]]; then
    # if not a new tab of tmux
    if [[ 0 < $(ps -ef | grep -e "#######" -e tmux | grep -v "#####" | wc -l) ]]; then
       exec tmux a
       exec tmux -2

Resources I have inspired from:

[1]     [ for detaching tmux instead exit ]
[2]  [Good one]



windows phone: offline readers

About Microsoft’s Devices

Nokia E-72
Nokia E-72

I have obtained a brand new Lumia 920. Actually it is a long story why I have chosen a Windows-based device of a company which I have hated as a software vendor. The point is, as per my long experiences, Microsoft is a great keyboard and mouse producer and which should stay that way.

I haven’t got any xbox experience. However, according to word of mouth, I have somehow grown the prejudice of a great device.

What I have seen, since 1992 starting with MS-DOS, have left a stinky smell of software experience. However just the contrary for hardware.

Within such a dilemma, addition of my brother’s incredible marketing of his windows phone and his experience, I just have given a shot.

My iPad still exists, but I wanted to see if I could have replaced it with this Lumia toy, with the confidence I have acquired by reading cover-to-cover two foreign language booksnd much much more articles on my E-72. (No, it was this century)

What I am looking for as a function in a device –whatever it is-, is to have the ability to read “things” even offline. That’s why I have bought an iPad 2 two years ago despite its really high cost. I have started to explore the applications of windows phone store for this purpose. Here what I have got lately:

The Applicatons

First I tried to search for the first application I (used to) use the most: Instapaper.

1) Stacks For Instapaper

Integrates with Instapaper.

Long story short: No, not this one. Just because it is trial, only 1 article to download from instapaper doesn’t make it real trial (2 in fact, but one of them is the manual). I don’t know how it behaves when large articles come. I know sounds like everything. However, no try no money. Not this one.

Line height option is good, eventhough.

Link to the application

2) NextGen Reader


I don’t know how the search matched this application but, this is rather an RSS reader with offline support. However I admired the “mobilizer” choice. This way you can choose to view the article simplified with one of Readability, Instapaper, getPocket, Google rendering engines as you wish. (bootstrap removers) Although could not understand what happens when, good feature anyway. I tried to choose readability engine but seems like, it just doesn’t download the readability version. It just puts a button to convert into Readability rendered versions “if” you have connection at that time. Anyway, ok.

However not happy with offline loading… It is (so to say) offline reading capable application but you have to get into the article to load the article really to have it offline. So, no offline mode is not OK.

Especially images… So much trouble. It doesn’t even state if the article is fully downloaded or not. You just have to check. If you have downloaded, then yes, you have the article with you. (It says “Caching images” when the app starts; but wait it doesn’t move anywhere, it takes forever)

Themes; ough! It doesn’t state however, if you want to change into another look, after changing, you must exit the application and start again to apply. Despite really great colors and 7 different themes, it should show me what I choose instantly.

Supports 5 fonts for now. Which contains my favorite Georgia. Fine.

Still have problems with resuming application. (you know the idea; go back to the application and go on from where you’ve left)

Not my type. But I guess because of the unique themes I just keep it.

Link to application

3) Reedabilityreedability

Integrates with Readability account.

I guess one of the best I have met on Windows Phone. Since I like the Readability renderer the most it looks to me the best tool.

Resuming speed is just great.

You can change the font one of 5 and size of the font. (Not line height). There are two themes; light and dark, as in real Readability.

Sharing a quote… I couldn’t do it, but seems OK for now.

Fast visuals.

Good to have.

Link to application

4) Pouch

Integrates with getpocket account.

It lacks of feedback. Just simple as that. You have to give feedback to the user, what you are doing right now. Really. I can’t understand if the application is trying to download or render or … whatever. Statusbars are filled and emptied and filled again and lots. And sometimes vibrates which means, well, I don’t know. (vibrate each time going to the main list?? really?)

Resuming just fails. Too much waiting. (I don’t know what happens next, just haven’t waited that much)

6 of fonts, and no Georgia or Garamond. Instead there is Times. Really?

Text to speech is a good idea. Reads the text to me. Really good one. However, I would rather choose “OK” button to close the font menu instead of builtin phone “back” touch button.

As to the benchmarks, Lumia 920 has a good CPU among the class. However nearly 2 seconds to open the template color dropbox made me this angry. The developer obviously wanted to put colors to the items, but functionality really lacks.

“View images”… If there is no image, you should test the “view image”. It’s fun to look at an empty space.


Link to the application with the hope of new update esp. for the reading experience rather than color (yes Georgia is one of them)

5) Phonly

Integrates with getpocket.

No offline capability and no promises at all.

So I just congratulate what it does for what it promises. It is a good tool reading getpocket articles on your account while online.

I just wanted to… Anyway.

Link to application

6) Swift Reader

Integrates to getpocket.

Despite saying some offline capability, it gives an error when I haven’t got any connection. Interesting.

If you like, you can read the titles of the articles offline.


Link to the application

7) Owl Readerowl

Integrates to getpocket.

Good one. Font selection, image handling, themes, resuming, automatic downloading, and great feedback of what is being done…

Just the one I was looking for. At least, until there is a rush into windows phone application development.

I got it. I use it. A few days more and I will buy it.

Link to the application

8) Comfortable Reader for Instapaper

Integrates to Instapaper.comfortable-reader

I will use it for a long time I suppose. It’s just simple and does what it promises. Auto updates are great. Live tile as the Owl Reader is also great.

There are two fonts; but yes, Georgia is there. So :) And there is a Sans too. Quick enough.

I have bought already. Trial was enough convincing for me.

Link to the application

9) Others

I have uninstalled at least 7 more applications which I cannot remember the names. I could have found. But there is time for them to be blamed.

Windows Phone market is not mature enough, there seems. But there is great hope. I admired the speed at most. There is long way to go though. So…

Oh, and one more,

10) Collectorcollector

Not sure what it is integrated to :) But though it deserves a try. Not my thing but it looks like a good effort was given into. At least there is something stable in here.

I keep it for a lonely day to try.

Link to the application

Turkish Holiday ICS Calendar File

Türkiye Resmi Tatil Günleri
Out Of Office olarak da işaretli bunlar.
Dosyayı buradan indirebilirsiniz [reklam filan yok]:
Bu da Google Calendar ICS adresi:
Aşağıdaki resim gibi oluyor.
Nasıl yapıldığını öğrenmek için, şuradaki öğreticiye bakabilirsiniz.

Turkish Holiday ICS Calendar File
Marked myself the out-of-office signatures.
Download the file here [direct link]:
This is the Google Calendar ICS:
It becomes like in the image below.
You could need a tutorial for how to use ICS calendars here is a tutorial.