Python 3.5.1 Syntax & APIs(Continue Updating..

print(x, end=' ')

instead of print(x) to escape the default line-changing-output.

print(str.ljust(size)) left-alignment with given size, samely, str.rjust(size) works just fine

 # Can you talk about the differences between the following commands?
 print('{}{}'.format(str1,str2))
 print('{1}{0}'.format(str1,str2))
 print('{sz1}{sz2}'.format(sz1=str1,sz2=str2))

 # using !a !s !r to cast conversion before print
 print('{}'.format('eels'))   #eels
 print('{!r}'.format('eels')) #'eels'

 # formatted output/floats, retain 3 digits after decimal point
 import math
 print('{0:.3f}'.format(math.pi))

 # % operator same as C/C++ usage is an older formatting
 print('%5.3f' % math.pi)

File reading & writing:

It seems that python do not support read&write file in one opened file, means flags like 'rw' returns an error, anyone whose test result differs from what I get please contact me ! note My python version is 3.5.1 64 bit. From python's official document I get little about this infomation, either. If no flag specified, default open mode is 'r'.

 # Open a file and assign it to f
 f=open('file','w+')
 f.write('somestring\n')
 f.close()
 with open('file','r+') as f:
     f.read()
     f.seek(0)
     f.readline()
     f.seek(0,0)# 0 = SEEK_SET, 1 = SEEK_CURRENT, 2 = SEEK_END

 f.closed

read() returns all contents out of the file, make sure the memory is large enough.

The with syntax automatically close the file f when running out of the seg of the code.

Saving structured data with json:

json stands for JavaScript Object Notation. The process taking python data hierarchies and convert to string representations called serialization.  Reconstructing the data from string representation is called deserialization. The string containing the data may store in a file or sent over net.

If you have an object 'x', view its JSON string representation with code:

 x = [1,'simple','test']
 import json
 json.dumps(x)
 with open('file','w') as f:
     json.dump(x,f)
 with open('file','r') as f:
     y = json.load(f)

This method can handle serialization of lists and dictionaries, but if to handle arbitrary class, use defined decoding method to serialize.

 import json
 def as_complex(dct)
     if '__complex__' in dct:
         return complex(dct['real'], dct['image'])
     return dct

 json.loads('{"__complex__": true, "real": 1, "image": 2}',
     object_hook=as_complex)
 # output (1 + 2j)

 import decimal
 json.loads('1.1', parse_float=decimal.Decimal)
 # output Decimal('1.1')

Use separators to eliminate space:

 import json
 json.dumps([1,2,3,{'': 5, '': 7}], separators=(',',':'),sort_keys=True)
 # output '[1,2,3,{'4':5,'6':7}]'

 #to test what I think
 print(json.dumps({'': 5, '': 7}, sort_keys=True,indent=4,separators=('~','n')))
 #output, here "indent" will append /n to every element
 #{
 #    "4"n5~
 #    "6"n7
 #}

If repeated keys exist in an object, loading the specfic object will ignore the older version of the key and keep the newer/last key & value.

 weird_json = '{"x": 1, "x": 2, "x": 3}'
 json.loads(weird_json)
 # {'x': 3}

Some JSON deserializer may set limits on:

• size of accepted JSON texts
• maximum level of nesting JSON objects & arrays
• number precision level
• content and maximum length of JSON strings

Defining Errors in Python by inherit class Exception:

 class Error(Exception):
     pass

 class InputError(Error):
     def __init__(self,expression,message):
         self.expression = expression
         self.message = message

 class TransitionError(Error):
     def __init__(self,previous,next,message):
         self.previous = previous
         self.next = next
         self.message = message

raise certain Error as an instance:

 raise InputError("Error1","line 11")

Unlike Java, naming the instance of error shall be like:

 except InputError as err
 # Now get the error Info
 print(err.expression, err.message)

Remember that Exceptions not caught in the try-except part will be thrown out before/after (it is uncertain) the finally part as errors. So we need to catch every Error the program would throw.

Some Objects like files will have pre-defined clean-up actions such as the with-syntax, are indicated in their doc.

In Python, 3 important kinds of methods are instance method, class method and static method. In class method and instance method, the first parameter represents the class or instance that is being mentioned.

 #!/usr/bin/python3
 class A(obj):
     # instance method
     def foo(self):
         print(self)
         pass
     # static method
     @staticmethod
     def foo2():
         pass
     # class method
     @classmethod
     def foo3(cls):
         print(cls)

Note the @ decorator is meaningful, not like @ in Java, it's a syntactic sugar.

 @staticmethod
 def foo2():
     pass

 # is equal to
 def foo2():
     pass
 foo2 = staticmethod(foo2)

 # the name after the decorator @ is a defined method name

Difference between the 3 kinds of methods:

• calling instance methods will pass the instance to the method as its first parameter
• calling class methods will pass the class to the method as its first parameter
• calling static methods will pass no parameter specific to class/instance to the method

Iterators:

 for element in [1, 2, 3]:
     print(element)
 for element in (1, 2, 3):
     print(element)
 for key in {'one':1, 'two':2}:
     print(key)
 for char in "":
     print(char)
 for line in open("myfile.txt"):
     print(line, end='')

Generator Exps:

 xv=[10,20,30]
 yv=[7,5,3]
 sum(x*y for x,y in zip(xv,yv)) # dot product
 max((x,y) for x,y in zip(xv,yv)) # maximum of x, comparably maximal y of pair (x,y)

Standard Lib:

os.getcwd(), os.chdir('subpath'), os.system('mkdir newfolder')

shutil.copy(src,dest), shutil.move(src,dest)

regular exps:

re.findall(r'\bf[a-z]*', 'which foot or hand fell fastest')

re.sub(r'(\b[a-z]+) \1', r'\1', 'cat in the the hat') # 'cat in the hat', means searching for two identical words and replace them with one

# same as ''cat in the the hat'.replace('the the', 'the')

math.cos(digi),math.log(digi,base)

random.choice(list), random.sample(list, count), random.random() # float

random.randrange(upperlimit)

statistics.mean(list), statistics.median(list), statistics.variance(list)

Check SciPy project for more numerical computation modules.

Internet Access:

 from urllib.request import urlopen
 with urlopen('http://tycho.usno.navy.mil/cgi-bin/timer.pl') as response:
     for line in response:
         line = line.decode('utf-8')  # Decoding the binary data to text.
         if 'EST' in line or 'EDT' in line:  # look for Eastern Time
             print(line)

 # works fine with ASCII coded page
 with urlopen('http://www.baidu.com') as res:
     for line in res:
         line = line.decode('utf-8')
             if 'GMT' in line:
                 f.write(line)

 server = smtplib.SMTP('http')
 server.sendmail('from','to',"'content'")
 server.quit()

Date & Time Module intros:

 from datetime import date
 now = date.today()
 now.strftime("%B...")

 birthday = date(2016,5,2)
 age = now - birthday

Python Strings & Bytes:

2 kinds of words: strings are used process words locally, bytes are used to transfer or store. If needed to use a word for processing, decode it to ascii or unicode, if needed to transfer/store, encode it to utf-8.

strings: 'abc', '\u0061\u0062\u0063', u'\u0061\u0062\u0063'

bytes: b'abc', b'\xe4\xb8\xad'

cast conversion: b->s : b'abc'.decode(/'ascii'); s->b: 'abc'.encode('utf-8')

Data Compression:

First note that only bytes have this method, not strings like 'abc' rather b'abc' fits.

import zlib

s=b'some string to be compressed'

t = zlib.compress(s)

zlib.decompress(t)

zlib.crc32(s)

other compress modules: gzip, bz2, lzma, zipfile, tarfile

Performance Measurement:

 #! /usr/bin/python3
 from timeit import Timer
 Timer('some work').timeit()

profile, pstats modules intros.

Quality Control:

doctest.testmod() # automatically test documentations ###sometest###

 def average(values):
     """Computes the arithmetic mean of a list of numbers.

     >>> print(average([20, 30, 70]))
     40.0
     """
     return sum(values) / len(values)

 import doctest
 doctest.testmod()

 """
 这里也可以写
 """
 def multiply(a,b):
     """
     >>> multiply(2,3)
     6
     >>> multiply('baka~',3)
     'baka~baka~baka~'
     """
     return a*b

 if __name__ == '__main__':
     import doctest
     doctest.testmod(verbose=True)

unittest module is more comprehensive in that it is maintained in a separate file.

 #! /usr/bin/python3
 # testunit.py
 # run in cmd by calling python testunit.py
 import unittest

 class MyNewTest(unittest.TestCase):
     def setUp(self):
         self.a = 1
     def testPass(self):
         self.a = self.a + 1
         self.assertEqual(self.a, 2)
     def testFail(self):
         self.a = self.a + 1
         self.assertEqual(self.a, 3)
     def test3(self):
         self.assertEqual(self.a, 1)
 if __name__ == '__main__':
     unittest.main()

Many other modules can be found, including xml, email, json, csv, sqlite3, gettext, locale, codecs.

Standard Lib - Part II:

reprlib:

use reprlib.repr to display a large or deeply nested containers with dots.

pprint: (pretty print which adds line breaks automatically)

 import pprint
 t = [[['black', 'cyan'], 'white', ['green', 'red']], [['magenta', 'yellow'], 'blue']]]
 pprint.pprint(t, width = 30)

textwrap: (deletes any \n in string & add \n every width, replace \r with ' ', replace \t with \n)

 import textwrap
 doc = """
 some doc texts
 """
 print(textwrap.fill(doc, width=40))

locale:

 import locale
 locale.setlocale(locale.LC_ALL,'English_United States.1252')
 conv = locale.localeconv() # get a mapping of conventions
 x = 1234567.8
 locale.format("%d", x, grouping = True)
 locale.format_string("%s%.*f", (conv['currency_symbol'], conv['frac_digits'], x), grouping = True)

Multi-threading:

 #! /usr/bin/python3
 import threading, zipfile
 class AsyncZip(threading.Thread):
     def __init__(self,infile,outfile):
         threading.Thread.__init__(self) # mark the instance as new thread
         self.infile = infile
         self.outfile = outfile
     def run(self):
         f = zipfile.ZipFile(self.outfile,'w',zipfile.ZIP_DEFLATED) # file, write, compress method
         f.write(self.infile)
         f.close()
         print('Finished background zip of:', self.infile)

 bg = AsyncZip('mydata.txt','myarchive.zip') # create instance and init its values
 bg.start() # start the thread from method run()
 print('The main continues to run in foreground.')

 bg.join() # wait for thread bg to finish
 print('Main program waited until bg was done.')

 i = threading.Thead.__init__
 i(bg) # prepares bg to be started again
 bg.start() # run again

 # compare to this (which works in a different way):
 bg.__init__('mydata.txt','myarchive.zip')
 bg.start()

Methods have return values, also classes have return values in python.

 # in method
 def m(value):
     return value**2

 # in class
 class c:
     def __init__(self,value):
         self.value = value # define & init member value
     def __repr__(self):
         return self.value**2

 >>> m(10)
 100
 >>> c(10)
 100

Weak Refs:

used in cascaded items. Delete once, all related fileds are also deleted if assigned as weak references.

 import weakref, gc
 class A:
     def __init__(self, value):
         self.value = value
     def __repr__(self):
         return str(self.value)

 a = A(10)                   # create a reference
 d = weakref.WeakValueDictionary()
 d['primary'] = a            # does not create a reference
 d['primary']                # fetch the object if it is still alive

 del a                       # remove the one reference
 gc.collect()                # run garbage collection right away

 d['primary']                # entry was automatically removed

array.array

collections.deque

bisect: # used to manipulate sorted lists; also known s.sort([reverse=True])

 import bisect
 scores = [(100, 'perl'), (200, 'tcl'), (400, 'lua'), (500, 'python')]
 bisect.insort(scores, (300, 'ruby'))
 scores
 # get [(100, 'perl'), (200, 'tcl'), (300, 'ruby'), (400, 'lua'), (500, 'python')]

... Only C/C++ have comma expression, java/python do not support it.

... from somelib import *

heapq:

 # heapq
 from heapq import heapify, heappop, heappush
 data=[1,3,5,7,9,2,4,6,8,0]
 heapify(data) # data is sorted in heap
 heappop(data) # from left to right, small to big pop
 heappush(data,-5) # add new entry in the right place, keeping the order

decimal:

 # This Decimal lib is not a C/C++ like decimal, it's more of a Java BigDecimal, can be used in financial applications
 # its precision can be adjusted by getcontext().prec by value
 from decimal import *
 Decimal('')/Decimal(7)
 # and more, like ln, exp, to_eng_string