We can put it this way; this is getting rid of archive images in wordpress.
Because sometimes for heavy sites, old images are just unneeded that much. Removing them is not the solution, we often look for.
So I prepared a script to resize to smaller size and reducing quality to take lower disk space.
Free to use, here it is:
This is from js console view of the Chrome:
Also check mailgun with telnet. Eggs’re there too.
I needed a simple cache solution using with flask and therefore used SimpleCache of werkzeug. I used to cache some json data returning from inner modules such as list of things, list of that, list of this; using a pattern similar to this one:
[Customer1.UserList.Page1] = {1:'bla', 2:'foo', 3:'bar'}
[Customer1.UserList.Page2] = {4:'bla', 5:'foo', 6:'bar'}
...
[Customer2.UserList.Page2] = {34:'bla', 35:'foo', 36:'bar'}
With a managable and smart expiry timeout (ie. 20 secs), this worked nice.
There were 3 problems in front of me, [1] on each action, I needed check if result is on cache, if not evaluate the action then push to cache, then code comes messy; so it should have had simpler usage [2] I needed to extend the timeout so that application is less busier overall, and [3] whenever a user makes a data manipulation, I needed to remove the keys with some specific pattern for the user.
Looking around, I found nearly nothing in terms of w/pattern key deleting.
At least, I found the LUA script sheerun shared on stackoverflow. Then wrote a method wrapper which would check the cache for me and if nothing there, evaluate the result and push it into the cache. So this section would be “more” automized.
This is the wrapper:
def getFromCacheOrSet(callable, *params, **opts):
key = opts.get('key', callable.__name__) + '.' + '.'.join(map(unicode, params)).encode('utf-8')
to = opts.get('timeout', 30)
extendTime = opts.get('extendTime', False)
try:
v = __cached.get(key)
if v is None:
v = callable(*params)
__cached.setex(key, to, v)
return v
else:
if extendTime: __cached.expire(key, to)
return v #fromCache
except Exception as e:
print('Error: get from cache fai
What it makes is that, if you would like to run a multiplication method such as multiply(4,16) you pass the method, its parameters to the wrapper so that it takes care of getting from the cache or evaluating. The point is wierdest line is obviously the “key” section which makes itself a key for the cache. Key is builded from the name of the method and its argument values.
For multiply(4, 16) key becomes “multiply.4.16″, if you rather would like to choose another string for this operation, you can pass it as key=”MultiplicationOp” in opts as in:
getFromCacheOrSet(multiply, 4, 16, key="MultiplicationOp", timeout=20) # 20 secs
then here we get this key on the cache:
MultiplicationOp.4.16 = 64
It evaluates and pushes this information to the cache.
This is not the best way by far. However, since I could not find a better way, it does the job that I have to settle. As I mentioned earlier, thanks to sheerun@stackoverflow I have something.
This is the removing part:
def removeFromCache(callable, *params, **opts):
try:
key = opts.get('key', callable.__name__) + '.' + '.'.join(map(unicode, params)).encode('utf-8')
__cached_del_keys(keys=[], args=[key + '*'])
except Exception as e:
print('Error: Couldnot delete from cache. Hopefully will expire itself soon.', e)
return False
What happens here is, deleting the matching keys fits the call with as we previously made for multiplication. Such as:
removeFromCache(multiply, 4, key="MultiplicationOp")
key value is mandatory only if you used it while setting the value (obviously). Warning should be this, in here there is no 2nd argument which get to be 16 here. This is because we would like to delete all “multiply.4.*” keys.
The LUA script which I call before the methods above:
__cached = redis.StrictRedis(host='localhost', port=6379, db=0) ##### db1 !
__DEL_LUA = ("""local keys = redis.call('keys', ARGV[1])
for i=1,#keys,5000 do
redis.call('del', unpack(keys, i, math.min(i+4999, #keys)))
end
return keys""")
__cached_del_keys = __cached.register_script(__DEL_LUA)
Overall usage, eg:
getFromCacheOrSet(getCustomerList, "customer_1", "arg1", "arg2", 3, timeout=1200) # 1200sec = 20 mins expire getFromCacheOrSet(getCustomerList, "customer_1", "arg1", "arg2", 2, timeout=1200) getFromCacheOrSet(getCustomerList, "customer_2", "arg5", "arg6", 1, timeout=1200)
## customer_1 does something to list, have to refresh removeFromCache(getCustomerList, "customer_1", "arg1") # remove all getCustomerList.customer_1.arg1.***
I hope anyone would be happy to use.
As a note; it’s better serializing the objects before pushing into the cache and deserializing respective text into object after getting from cache server. These might help:
def serialize(obj):
return base64.b64encode(pickle.dumps(obj))
def deserialize(s):
return pickle.loads(base64.b64decode(s))
References:
1. redis-py
2. redis
3. werkzeug SimpleCache and RedisCache
4. flask
5. issue on stackoverflow
I have had the problem authentication with my ssh server with a private key. It asks password despite no password needed. Here is the log:
mememe@Mac:~$ ssh -v -i ~/.ssh/id.ppk root@remote.machine.com
OpenSSH_6.2p2, OSSLShim 0.9.8r 8 Dec 2011
debug1: Reading configuration data /Users/<username>/.ssh/config
debug1: Reading configuration data /etc/ssh_config
debug1: /etc/ssh_config line 20: Applying options for *
debug1: /etc/ssh_config line 102: Applying options for *
debug1: Connecting toremote.machine.com [999.1.1.1] port 22.
debug1: Connection established.
debug1: identity file /Users/<username>/.ssh/id.ppk type -1
debug1: identity file /Users/<username>/<username>/id.ppk-cert type -1
debug1: Enabling compatibility mode for protocol 2.0
debug1: Local version string SSH-2.0-OpenSSH_6.2
debug1: Remote protocol version 2.0, remote software version OpenSSH_6.6.1p1 Ubuntu-2ubuntu2
debug1: match: OpenSSH_6.6.1p1 Ubuntu-2ubuntu2 pat OpenSSH*
debug1: SSH2_MSG_KEXINIT sent
debug1: SSH2_MSG_KEXINIT received
debug1: kex: server->client aes128-ctr hmac-md5-etm@openssh.com none
debug1: kex: client->server aes128-ctr hmac-md5-etm@openssh.com none
debug1: SSH2_MSG_KEX_DH_GEX_REQUEST(1024<1024<8192) sent
debug1: expecting SSH2_MSG_KEX_DH_GEX_GROUP
debug1: SSH2_MSG_KEX_DH_GEX_INIT sent
debug1: expecting SSH2_MSG_KEX_DH_GEX_REPLY
Host sfs
debug1: Server host key: RSA 43:xx:72:xx:xx:b0:xx:8a:xx:xx:xx:xx:xx:xx:a7
debug1: Host 'remote.machine.com' is known and matches the RSA host key.
debug1: Found key in /Users/<username>/.ssh/known_hosts:5
debug1: ssh_rsa_verify: signature correct
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug1: SSH2_MSG_NEWKEYS received
debug1: Roaming not allowed by server
debug1: SSH2_MSG_SERVICE_REQUEST sent
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug1: Authentications that can continue: publickey
debug1: Next authentication method: publickey
debug1: Trying private key: /Users/<username>/.ssh/id.ppk
debug1: key_parse_private_pem: PEM_read_PrivateKey failed
debug1: read PEM private key done: type <unknown>
debug1: No more authentication methods to try.
Permission denied (publickey).
So it appears that, I tried to use PuTTY gen generated key file with OpenSSH. Therefore, without windows I had to use the steps:
1) This installs putty along with puttygen: (MacPorts needed)
$ sudo port install putty
2) This converts my ssh keygen into OpenSSH key type:
$ puttygen id.ppk -O private-openssh -o id_openssh
now done. let’s go:
$ ssh -i ~/.ssh/id_openssh root@remote.machine.com
goes ok for me.
There are nice samples for pair programming solutions. Well, I have seen a few, too.
I needed a simpler and quicker solution so here I went like this:
add to end of /etc/bash.bashrc:
exit() {
if [[ -z $TMUX ]]; then
builtin exit
else
tmux detach
fi
}
close() {
builtin exit
}
if [[ "x" == "x$TMUX" ]]; then
# if not a new tab of tmux
if [[ 0 < $(ps -ef | grep -e "#######" -e tmux | grep -v "#####" | wc -l) ]]; then
exec tmux a
else
exec tmux -2
fi
fi
Resources I have inspired from:
[1] https://coderwall.com/p/powgbg [ for detaching tmux instead exit ]
[2] http://collectiveidea.com/blog/archives/2014/02/18/a-simple-pair-programming-setup-with-ssh-and-tmux/ [Good one]
[3] http://superuser.com/questions/456187/connecting-a-tmux-pane-to-a-remote-server
Adventures of Foo & Bar 