David's Wiki

Python: Difference between revisions

← Older edit
 
(58 intermediate revisions by the same user not shown)
Line 58: Line 58:
print(f'Hello {name}! This is {program}')
print(f'Hello {name}! This is {program}')


print("%s %s" %('Hello','World',))
print("%s %s" % ('Hello','World',))


name = 'world'
name = 'world'
program ='python'
program ='python'
print('Hello {name}!This is{program}.'.format(name=name,program=program))
print('Hello {}! This is {}.'.format(name, program))
print('Hello {name}! This is {program}.'.format(name=name, program=program))
 
# Format to two decimal places
print(f"Accuracy: {accuracy:.02f}%")
 
# Format an int to 2 digits
print(f"Size: {size:02}%")
</syntaxhighlight>
</syntaxhighlight>


Line 68: Line 75:
Use Numpy to provide array functionality
Use Numpy to provide array functionality
====Array Indexing====
====Array Indexing====
[https://docs.scipy.org/doc/numpy/user/basics.indexing.html Scipy Reference]
[https://numpy.org/doc/stable/user/basics.indexing.html Numpy Indexing]
Numpy has very powerful indexing. See the above reference.


===Filesystem===
===Filesystem===
Line 98: Line 106:
pp = Path(p, "files.tar.gz")
pp = Path(p, "files.tar.gz")


pp.suffix # returns ".gz"
pp.suffix     # returns ".gz"
pp.suffixes # returns [".tar", ".gz"]
pp.suffixes   # returns [".tar", ".gz"]
pp.name # returns "files.tar.gz"
pp.name       # returns "files.tar.gz"
pp.parent # returns "my_folder"
pp.parent     # returns "my_folder"
</syntaxhighlight>
</syntaxhighlight>
;Notes
;Notes
Line 111: Line 119:
[https://stackoverflow.com/questions/3207219/how-do-i-list-all-files-of-a-directory Reference]
[https://stackoverflow.com/questions/3207219/how-do-i-list-all-files-of-a-directory Reference]
<syntaxhighlight lang="python">
<syntaxhighlight lang="python">
gazeDir = "Gaze_txt_files"
gaze_directory = "Gaze_txt_files"
# List of folders in root folder
# List of folders in root folder
gazeFolders = [path.join(gazeDir, x) for x in os.listdir(gazeDir)]
gaze_folders = [path.join(gaze_directory, x) for x in os.listdir(gaze_directory)]
# List of files 2 folders down
# List of files 2 folders down
gazeFiles = [path.join(x, y) for x in gazeFolders for y in os.listdir(x)]
gaze_files = [path.join(x, y) for x in gaze_folders for y in os.listdir(x)]
</syntaxhighlight>
</syntaxhighlight>
See also glob.


====Read/Write entire text file into a list====
====Read/Write entire text file into a list====
Line 170: Line 180:
<syntaxhighlight lang="python">
<syntaxhighlight lang="python">
import re
import re
myReg = re.compile(r'height:(\d+)cm')
my_regex = re.compile(r'height:(\d+)cm')
myMatch = myReg.match("height:33cm");
my_match = my_regex.match("height:33cm");
print(myMatch[1])
print(my_match[1])
# 33
# 33
</syntaxhighlight>
</syntaxhighlight>
;Notes
;Notes
* <code>re.match</code> will return None if there is no match
* <code>re.match</code> will return None if there is no match
Line 225: Line 236:
         'api_paste_code':source_code,  
         'api_paste_code':source_code,  
         'api_paste_format':'python'}  
         'api_paste_format':'python'}  
 
 
# sending post request and saving response as response object  
# sending post request and saving response as response object  
r = requests.post(url = API_ENDPOINT, data = data)  
r = requests.post(url = API_ENDPOINT, data = data)  
 
 
# extracting response text
# extracting response text
pastebin_url = r.text  
pastebin_url = r.text  
print("The pastebin URL is:%s"%pastebin_url)  
print("The pastebin URL is:%s"%pastebin_url)  
Line 255: Line 266:


===if main===
===if main===
[https://stackoverflow.com/questions/419163/what-does-if-name-main-do What does if __name__ == "__main__" do?]
If you are writing a script with functions you want to be included in other scripts, use <code>__name__</code> to detect if your script is being run or being imported.
If you are writing a script with functions you want to be included in other scripts, use <code>__name__</code> to detect if your script is being run or being imported.
[https://stackoverflow.com/questions/419163/what-does-if-name-main-do What does if __name__ == "__main__" do?]
<syntaxhighlight lang="python">
<syntaxhighlight lang="python">
if __name__ == "__main__":
if __name__ == "__main__":
  # do..something..here
</syntaxhighlight>
</syntaxhighlight>


===iterators and iterables===
Iterables include lists, np arrays, tuples. 
To create an iterator, pass an iterable to the <code>iter()</code> function.
<syntaxhighlight lang="python">
my_arr = [1,2,3,4]
my_iter = iter(my_arr)
v1 = my_iter.next()
</syntaxhighlight>


===iterators and iterables===
<code>itertools</code> contains many helper functions for interacting with iterables and iterators.
iterators


====zip====
====zip====
Line 278: Line 299:


i.e. enumerate([a1,...], start=0) = [(0, a1), (1, a2), ...]
i.e. enumerate([a1,...], start=0) = [(0, a1), (1, a2), ...]
====slice====
<code>itertools.islice</code> will allow you to create a slice from an iterable
<syntaxhighlight lang="python">
from itertools import islice
import numpy as np
a = np.arange(5)
b = islice(a, 3)
list(b) # [0,1,2]
</syntaxhighlight>
===Exceptions===
See [https://docs.python.org/3/library/exceptions.html https://docs.python.org/3/library/exceptions.html]
;Raising
<syntaxhighlight lang="python">
raise ValueError("You have bad inputs")
assert 1=1, "Something is very wrong if 1!=1"
</syntaxhighlight>
;Try Catch/Except
<syntaxhighlight lang="python>
try:
  something_which_may_raise()
except AssertError as error:
  do_fallback()
  raise # Raise the previous error.
else:
  do_something_if_no_exception()
finally:
  finish_program_and_cleanup()
</syntaxhighlight>


==Classes==
==Classes==
===Static and Class methods===
See [https://realpython.com/instance-class-and-static-methods-demystified/ realpython]
<syntaxhighlight lang="python">
<syntaxhighlight lang="python">
class Car:
class MyClass:
  def __init__(self):
    def method(self):
        return 'instance method called', self
 
    @classmethod
    def classmethod(cls):
        return 'class method called', cls
 
    @staticmethod
    def staticmethod():
        return 'static method called'
</syntaxhighlight>
</syntaxhighlight>
;Notes
* That the Google Python style guide discourages use of static methods.
** Class methods should only be used to define alternative constructors (e.g. from_matrix).
==Multithreading==
===threading===
[https://docs.python.org/3/library/threading.html?highlight=threading#module-threading <code>import threading</code>]
Use <code>threading.Thread</code> to create a thread.
===concurrrency===
In Python 3.2+, [https://docs.python.org/3/library/concurrent.futures.html#module-concurrent.futures <code>concurrent.futures</code>] gives you access to thread pools.
<syntaxhighlight lang="python">
import os
import threading
from concurrent.futures import ThreadPoolExecutor, as_completed
executor = ThreadPoolExecutor(max_workers=os.cpu_count())
thread_lock = threading.Lock()
total = 0
def do_something(a, b):
  with thread_lock:
    total += a + b
  return total
my_futures = []
for i in range(5):
  future = executor.submit(do_something, 1, 2+i)
  my_futures.append(future)
for future in as_completed(my_futures):
  future.result()
executor.shutdown()
</syntaxhighlight>
* <code>len(os.sched_getaffinity(0))</code> returns the number of threads available to the Python process.
* Starting in Python 3.5, if <code>max_workers</code> is none, it defaults to <code>5 * os.cpu_count()</code>.
** <code>os.cpu_count()</code> returns the number of logical CPUs (i.e. threads)
* <code>executor.shutdown()</code> will wait for all jobs to finish but you cannot submit any additional jobs from other threads, after calling shutdown.
* List operations are thread-safe but most other operations will require using a thread lock or semaphore.


==Data Structures==
==Data Structures==
===Tuples===
Tuples are immutable lists. This means that have fixed size and fixed elements, though elements themselves may be mutable.
In general, they perform marginally faster than lists so you should use tuples over lists when possible, especially as parameters to functions.
Typically people use tuples as structs, i.e. objects with structure such as coordinates. See [https://stackoverflow.com/questions/626759/whats-the-difference-between-lists-and-tuples StackOverflow: Difference between lists and tuples].
<syntaxhighlight lang="python">
# Tuple with one element
m_tuple = (1,)
# Tuple with multiple elements
vals = (1,2,3, "car")
# Return a tuple
def int_divide(a, b):
  return a // b, a % b
</syntaxhighlight>
===Lists===
===Lists===
The default data structure in Python is lists.<br>
The default data structure in Python is lists.<br>
Line 332: Line 461:
for k, v in my_map.items():
for k, v in my_map.items():
   print(k, v)
   print(k, v)
</syntaxhighlight>
==Numpy==
{{main | NumPy}}
See also Cupy which is a numpy interface implemented with CUDA for GPU acceleration. Large speedups can be had for big arrays.
===random===
Legacy code uses functions from <code>np.random.*</code>.
New code should initialize a rng using <code>np.random.default_rng()</code>. 
See [https://numpy.org/doc/stable/reference/random/generator.html Random Generator] for more details.
<syntaxhighlight lang="python">
import numpy as np
rng = np.random.default_rng()
# Random integer between [0, 6)
rng.integers(0, 6)
# array of 5 random integers
rng.integers(0, 6, size=5)
</syntaxhighlight>
</syntaxhighlight>


Line 370: Line 520:
# Write to file
# Write to file
with open("my_data.json", "w") as f:
with open("my_data.json", "w") as f:
   json.dump(my_data, f)
   json.dump(my_data, f, indent=2)


# Read from file
# Read from file
Line 379: Line 529:
; Notes
; Notes
* Using <code>json.dump(data, f)</code> will dump without pretty printing
* Using <code>json.dump(data, f)</code> will dump without pretty printing
** Add indent parameter for pretty printing.
==Type Annotations==
Python 3 supports adding type annotations. However it is not enforced at runtime. 
You can check types ahead of time using [https://google.github.io/pytype/ pytype].


<syntaxhighlight lang="python">
function add_two_values(a: float, b: float) -> float:
    return a + b
</syntaxhighlight>
==Images==
===Pillow (PIL)===
===Pillow (PIL)===
<code>pip install pillow</code>
<code>pip install pillow</code>


====Images====
<syntaxhighlight lang="python">
<syntaxhighlight lang="python>
from PIL import Image, ImageOps
from PIL import Image
from


img = Image.open("my_image.png")
img = Image.open("my_image.png")
Line 392: Line 551:
img = np.array(img)
img = np.array(img)
</syntaxhighlight>
</syntaxhighlight>
* <code>flip()</code> - flips across y axis
 
* <code>mirror()</code> - flips across x axis
* <code>ImageOps.flip(img)</code> - Returns an image flipped across y axis
* <code>ImageOps.mirror(img)</code> - Returns an image flipped across x axis
 
===Bilinear Interpolation===
Coped from [https://stackoverflow.com/questions/12729228/simple-efficient-bilinear-interpolation-of-images-in-numpy-and-python https://stackoverflow.com/questions/12729228/simple-efficient-bilinear-interpolation-of-images-in-numpy-and-python]
{{ hidden | Bilinear Interpolation function |
<syntaxhighlight lang="python">
def bilinear_interpolate(im, x, y):
    """
    Basic bilinear interpolation
    :param im:
    :param x:
    :param y:
    :return:
    """
    x = np.asarray(x)
    y = np.asarray(y)
 
    x0 = np.floor(x).astype(int)
    x1 = x0 + 1
    y0 = np.floor(y).astype(int)
    y1 = y0 + 1
 
    x0 = np.clip(x0, 0, im.shape[1] - 1)
    x1 = np.clip(x1, 0, im.shape[1] - 1)
    y0 = np.clip(y0, 0, im.shape[0] - 1)
    y1 = np.clip(y1, 0, im.shape[0] - 1)
 
    Ia = im[y0, x0]
    Ib = im[y1, x0]
    Ic = im[y0, x1]
    Id = im[y1, x1]
 
    wa = (x1 - x) * (y1 - y)
    wb = (x1 - x) * (y - y0)
    wc = (x - x0) * (y1 - y)
    wd = (x - x0) * (y - y0)
   
    if len(Ia.shape) > len(wa.shape):
        wa = wa[..., np.newaxis]
        wb = wb[..., np.newaxis]
        wc = wc[..., np.newaxis]
        wd = wd[..., np.newaxis]
 
    return wa * Ia + wb * Ib + wc * Ic + wd * Id
</syntaxhighlight>
}}


==Libraries==
==Libraries==
===Numpy===
Other notable libraries.
 
===Matplotlib===
===Matplotlib===
{{main | Matplotlib}}
{{main | Matplotlib}}
Line 405: Line 611:
Alternatively, there are also Python bindings for ggplot2<br>
Alternatively, there are also Python bindings for ggplot2<br>


===Pillow (PIL)===
===configargparse===
<code>pip install pillow</code>
[https://pypi.org/project/ConfigArgParse/ ConfigArgParse] is the same as argparse except it allows you to use config files as args.
 
<syntaxhighlight lang="python">
parser = configargparse.ArgParser()
parser.add('-c', '--config', is_config_file=True, help='config file path')
 
# Parse all args, throw exception on unknown args.
parser.parse_args()
 
# Parse only known args.
parser.parse_known_args()
</syntaxhighlight>
 
If you want to use bools without store-true or store-false, you need to define an str2bool function:
[https://stackoverflow.com/questions/15008758/parsing-boolean-values-with-argparse Stack Overflow Answer]
{{ hidden | str2bool |
<syntaxhighlight lang="python">
def str2bool(val):
  """Converts the string value to a bool.
  Args:
    val: string representing true or false
  Returns:
    bool
  """
  if isinstance(val, bool):
    return val
  if val.lower() in ('yes', 'true', 't', 'y', '1'):
    return True
  elif val.lower() in ('no', 'false', 'f', 'n', '0'):
    return False
  else:
    raise argparse.ArgumentTypeError('Boolean value expected.')
 
#...
parser.add_argument("--augment",
                    type=str2bool,
                    help="Augment",
                    default=False)
</syntaxhighlight>
}}


This is a very useful library for loading images.
[[Category:Programming languages]]
You can also load images using matplotlib, OpenCV, or Tensorflow.
Retrieved from "https://wiki.davidl.me/view/Python"